environment setup

 2.Install Software

1. JDK

Install JDK. Currently, JDK 8 is used.

http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

1. Find out the path JDK 1.8 is installed

   $ which java /Library/Java/JavaVirtualMachines/jdk1. 8 .0_112.jdk/Contents/Home/bin/java

2. Set environment variables

   $ cd ~ $ vi .bash_profile ... export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1. 8 .0_112.jdk/Contents/Home PATH=$JAVA_HOME/bin:$PATH    :wq! $ source .bash_profile

3. Check Java version

   $ java -version java version  "1.8.0_112" Java(TM) SE Runtime Environment (build  1.8 .0_112-b16) Java HotSpot(TM)  64 -Bit Server VM (build  25.112 -b16, mixed mode)

2. Git

Install git:  https://git-scm.com/downloads

FYI: http://learngitbranching.js.org/  if you want to know more about git commands

1. Check if git is installed successfully

   $ git --version 15 : 19 : 37.961988  git.c: 344                trace: built-in: git  'version' git version  2.7 . 4  (Apple Git- 66 )

2. Define basic git config settings

   git config --global user.name  "[your userid]" git config --global user.email  "[your userid]@coupang.com" git config --global core.precomposeunicode  true git config --global core.autocrlf  false git config --global core.filemode  false

3. Database (MySQL)

1. Download and Setup
   OS X: Local development Divi install - Mac (Local DB Installation for Mac OS)

2. Setup in ~/.bash_profile file

   alias mysql=/usr/local/mysql/bin/mysql alias mysqladmin=/usr/local/mysql/bin/mysqladmin export PATH=$PATH:/usr/local/mysql/bin

3. DB user, schemas setup (ods, fds)          

Need to log in as root user to perform the following steps.

• Log in as root user:

mysql -u root -p'password'       (replace with actual root password)

OR

mysql -u root -p                        (type in password once prompt)

• Create Users

Create Users: CREATE USER  'coupang' @ 'localhost'  IDENTIFIED BY  'test' ; CREATE USER  'winter' @ 'localhost'  IDENTIFIED BY  'test' ; CREATE USER  'summer' @ 'localhost'  IDENTIFIED BY  'test' ; Create databases: CREATE DATABASE `fds` DEFAULT CHARACTER SET utf8 COLLATE utf8_bin; CREATE DATABASE `ods` DEFAULT CHARACTER SET utf8 COLLATE utf8_bin; Grant Permissions: USE mysql; INSERT INTO db (HOST, Db, USER, Select_priv, Insert_priv, Update_priv, Delete_priv, Create_priv, Drop_priv, Index_priv, Alter_priv)      VALUES( 'localhost' , 'fds' , 'coupang' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' ); INSERT INTO db (HOST, Db, USER, Select_priv, Insert_priv, Update_priv, Delete_priv, Create_priv, Drop_priv, Index_priv, Alter_priv)     VALUES( 'localhost' , 'ods' , 'coupang' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' );      INSERT INTO db (HOST,Db,USER,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv,Index_priv, Alter_priv)         VALUES( 'localhost' , 'fds' , 'winter' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' );      INSERT INTO db (HOST,Db,USER,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv,Index_priv, Alter_priv)         VALUES( '10.%.%.%' , 'fds' , 'winter' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' );      INSERT INTO db (HOST,Db,USER,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv,Index_priv, Alter_priv)         VALUES( 'localhost' , 'fds' , 'summer' , 'Y' , 'Y' , 'Y' , 'Y' , 'N' , 'N' , 'N' , 'N' );      INSERT INTO db (HOST,Db,USER,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv,Index_priv, Alter_priv)         VALUES( '10.%.%.%' , 'fds' , 'summer' , 'Y' , 'Y' , 'Y' , 'Y' , 'N' , 'N' , 'N' , 'N' );      INSERT INTO db (HOST,Db,USER,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv,Index_priv, Alter_priv)         VALUES( 'localhost' , 'ods' , 'winter' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' );      INSERT INTO db (HOST,Db,USER,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv,Index_priv, Alter_priv)         VALUES( '10.%.%.%' , 'ods' , 'winter' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' );      INSERT INTO db (HOST,Db,USER,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv,Index_priv, Alter_priv)         VALUES( 'localhost' , 'ods' , 'summer' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' );      INSERT INTO db (HOST,Db,USER,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv,Index_priv, Alter_priv)         VALUES( '10.%.%.%' , 'ods' , 'summer' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' , 'Y' );   FLUSH PRIVILEGES;

Note: if the insert statement execution doesn’t work, don’t copy/paste.  Type it out.  It may be related to the single quote issue.

In case you need to uninstall MySQL, you can follow the Solution at the bottom of this website. http://community.jaspersoft.com/wiki/uninstall-mysql-mac-os-x

4. Tomcat

1. Download Tomcat
   http://tomcat.apache.org/download-80.cgi
   Click the tar.gz link under "Core" section.
    

2. Extract it at the folder you prefer. (e.g. /usr/local/) 

   tar -zxvf apache-tomcat- 8.5 . 9 .tar

3. Setup links

   sudo rm -f /Library/Tomcat sudo ln -s /usr/local/apache-tomcat- 8.5 . 9  /Library/Tomcat sudo chown -R [the userid you setup on Mac] /Library/Tomcat sudo chmod +x /Library/Tomcat/bin/*.sh

4. Setup environment variables in ~/.bash_profile file

   export CATALINA_HOME=/usr/local/apache-tomcat- 8.5 . 9 PATH=$CATALINA_HOME/bin:$JAVA_HOME/bin:$PATH

5. Commands to start/stop servers

   /Library/Tomcat/bin/startup.sh /Library/Tomcat/bin/shutdown.sh

5. IntelliJ

NOTE:  If you are out of office, make sure you connect to VPN environment.

1. Download IntelliJ
   Http://www.jetbrains.com/idea/download/index.html 
   
   Download the Ultimate Edition.  (Note:  I downloaded the version 14.x.x.  I also download the latest version (15.x) but it does't work when I inserted the License Server)
2. Apply License Activation
   In the first screen "Register Click" 
   Open or the development environment IntelliJ then Menu> Help> Register Click
   A popup will display with title "IntelliJ License Server"
   Select "License Server" radio button.
   License Server Address:  http://xxx()

3. Setup IntelliJ VM properties
   Edit the file /Applications/IntelliJ IDEA 14.app/Contents/bin/idea.vmoptions
   Make changes as shown below

   -Xms128m -Xmx2048m -XX: MaxPermSize = 350m -XX:ReservedCodeCacheSize=96m -XX:+UseCodeCacheFlushing -she -Dsum.io.useCanonCache= false -Djava.net.preferIPv4Stack= true

   After saving the changes, copy this file to ~/Library/Preferences/IntelliJIdeaXX/idea.vmoptions

4. Clone the (DFS) project
   After installation, you can create project by checking out from Version Control.  (If you are in the Welcome to IntelliJ IDEA popup, you may select clone the project from Version Control.)
   Select git.
   Fill out the information in the Clone Repository Popup

   
   
   In case there is a popup of 'Import Project', select 'Gradle'.  Click <OK> to take the default 
   
   File > Project > Project SDK: set to JDK 1.8 && Project Language Level:  Set to 8.0 
   
   
   

5. Configurations
   After your project is imported/cloned successfully AND it connects to git version control, perform the followings:
   
   IntelliJ IDEA > Preferences
   
   Select Plugins.  Install the followings:
     Lombok
     Python
     Eclipse Code Formatter
     Handlebars/Mustache
   
   After installation, it will ask you to restart IntelliJ.  Go ahead to restart IntelliJ.
   
   
   
   Select Compiler and update the followings:
      Resource Patterns:  Add "logback*.groovy"
      Build Process Heap Size: 1024
   
   
   Select 'Annotation Processors' and check the checkbox of 'Enable annotation processing'
   
   
   
   Select 'File Encodings' and modify the followings:
     Default encoding for properties files:  UTF-8
     Transparent native-to-ascii conversion:  Checked
   
   
   Select 'Gradle' and update the followings:
     Gradle VM options:  -Dfile.encoding=UTF-8
     Use auto-import:  Checked
   
   
   Search 'File and Code Templates'.  Select 'Includes' tab and select 'File Header'.  Update the following:
   
   

   /**   * User: your userid (your email address)   * Date: ${DATE}   * Time: ${TIME}   */

6. Go to terminal window.  Open the file /etc/hosts.  Add the following line:

   127.0.0.1       local.coupang.com

6.Installing RabbitMQ

The following step is required for running the development server. The instruction seems to require installing Erlang, but I noticed it was already included using the homebrew method to install RabbitMQ.

• RabbitMQ Installation Guide at Local
• Verify installation http://127.0.0.1:15672/#/ using guest/guest.

You will see the following error if RabbitMQ is not setup when running tomcat.

Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private com.coupang.rabbitkeeper.RabbitKeeper

6.1 Create Local RabbitMQ user

cd $RABBITMQ_HOME/$VERSION/sbin

rabbitmqctl add_user coupang coupang
rabbitmqctl set_user_tags coupang administrator
rabbitmqctl set_permissions -p / coupang ".*" ".*" ".*"

7.key.dat

Unzip the file.  For the local environment, put the unzipped file under the path /usr/local/coupang/.  Read/write permissions at the 'coupang' directory is required.

Note:  "root" is used to create the folder 'coupang' and own the file.

3.DB table & testing data setup

Run below gradle tasks one by one to create table and testing data:

./gradlew :fraud-detector-domain:flywayClean -Pprofile.flyway=develop
./gradlew :fraud-detector-domain:flywayInit -Pprofile.flyway=develop
./gradlew :fraud-detector-domain:flywayMigrate -Pprofile.flyway=develop

 

4.FDS Tomcat server configuration

1, IntelliJ IDEA > Edit Configurations > Click + on the left top > Select Tomcat Server(Local) 

2, Server tab configuration

• Name: any unique name is ok, e.g. FDS

• Application server:  Configure to the tomcat server which you installed previously

• VM options: 

  -Dfile.encoding=UTF- 8 -Xmx1024m -XX:PermSize=64m -XX:MaxPermSize=256m -Dspring.profiles.active=integration  -Dasset.timestamp.generator=LOCAL_TIME -Dconfig.override=classpath:vitamin/properties/project-develop-override.xml

 

 

 

3, "Deployment" tab configuration

• Click + at "Deploy at the server startup"

• Select "Artifact..."

• Select "fraud-detector-interfaces.war(exploded)"

• Click "OK"

4, Start server

When server is up, go to http://local.coupang.com:8080/

Issue: For newer IntelliJ version ( 2016), you might run into the problem of local hostname unknown. Simply using this command to fix it: 

echo "127.0.0.1 $HOSTNAME" | sudo tee -a /etc/hosts

5.Batch configurations

example to configure two key batches

1. SpringMainBatch

• Click Run > Edit Configurations
• Click + at the left top of the pop up window
• Select 'Application'
• Fill followings to the pop up window

               Name: SpringMainBatch

               Main class: com.coupang.batch.SpringBatchManager

               VM options:

-Xms256m -Xmx1024m -XX:MaxPermSize=512m  -Dspring.profiles.active=integration -DbatchContextConfig=com.coupang.frauddetector.domain.batch.configuration.FraudDetectorBatchContextConfig -Dconfig.override=classpath:vitamin/properties/project-integration-override.xml

• Program arguments: classpath:/jobs/ruleprocessor/batch-job.xml

 

 

       2. SpringMainBatchFrequent

• Click Run > Edit Configurations
• Click + at the left top of the pop up window
• Select 'Application'
• Fill followings to the pop up window

               Name: SpringMainBatchFrequent

               Main class: com.coupang.batch.SpringBatchManager

               VM options:

-Xms256m -Xmx1024m -XX:MaxPermSize=512m  -Dspring.profiles.active=develop -DbatchContextConfig=com.coupang.frauddetector.domain.batch.configuration.FraudDetectorBatchContextConfig  -Dconfig.override=classpath:vitamin/properties/project-develop-override.xml

• Program arguments: classpath:/jobs/ruleprocessor/batch-job-frequent.xml

 

6.Learning Materials

• Deployment process
• Deployment Tools
• PSI plan & Iteration Release Plan
• FDS Environment Info
• FDS Architecture & Business case
• Find out more in FDS team wiki, US wiki