How to setup CI development environment in Linux

In the past 2 days, I setup a CI development environment in Linux:

Git + Jenkins + Maven(Nexus)  + Tomcat

I write the steps down to keep memo.

Pre-installed software:

1. Java: I use JDK1.8.0_25

http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

jdk-8u25-linux-i586.rpm

Install the jdk, and add the environment variable:

vi /etc/profile

JAVA_HOME=/usr/java/jdk1.8.0_25
export JRE_HOME=/usr/java/jdk1.8.0_25
export JRE_BIN=/usr/java/jdk1.8.0_25/bin
export CLASSPATH=$JRE_HOME/lib:$CLASSPATH
export PATH=$JRE_HOME/bin:$PATH
source /etc/profile

Verify: in the terminal, run: java -version and could see the version

2. Maven:

http://maven.apache.org/download.cgi

apache-maven-3.2.3-bin.tar.gz

extract the tar file and move it to /usr/local:

tar -xvf apache-maven-3.2.3-bin.tar.gz

mv apache-maven-3.2.3 /usr/local/apache-maven-3.2.3

add environment variable:

vi /etc/profile

MAVEN_HOME=/usr/local/apache-maven-3.2.3
export MAVEN_HOME
export PATH=${PATH}:${MAVEN_HOME}/bin

source /etc/profile

Verify: in the terminal, run: mvn -v and could see the version

3. Git

IMPORTANT: In redhat 6, and you use yum install git you could only install 1.7.1 version, but that version is out date, and has some problem for authentication, so if you install the git 1.7.1, you need to uninstall it, using rpm -e git

and maybe ask you uninstall the perl-git something, anyway, you need to rpm -e git per-get to remove them all.

Finally, I find a document about how to install latest version of git:

http://tecadmin.net/install-git-1-9-on-centos-rhel/

Step 1: Install Required Packages

yum install curl-devel expat-devel gettext-devel openssl-devel zlib-devel gcc perl-ExtUtils-MakeMaker

Step 2: Download and Compile Git Source

# cd /usr/src
# wget https://www.kernel.org/pub/software/scm/git/git-1.9.4.tar.gz
# tar xzf git-1.9.4.tar.gz

Step 3. Check Git Version

One completion of above steps, you have successfully install Git in your system. Let use following command to check git version

# git --version

git version 1.9.4

4. Jenkins:

http://pkg.jenkins-ci.org/redhat-stable/

jenkins-1.580.1-1.1.noarch.rpm

install the rpm, and then startup jenkins by:

service jenkins start

and visit:

http://localhost:8080

you should see the jenkins UI.

chmod -R 777 /var/lib/jenkins

5. Tomcat:

http://tomcat.apache.org/download-70.cgi

apache-tomcat-7.0.57.tar.gz

extract the tar file and move it to /usr/local:

tar -xvf apache-tomcat-7.0.57.tar.gz

mv  apache-tomcat-7.0.57 /usr/local/apache-tomcat-7.0.57

Make sure you have the permission to change the folder, otherwise, change the permission to 777

chmod -R 777 /usr/local/apache-tomcat-7.0.57

modify /usr/local/apache-tomcat-7.0.57/conf/tomcat-users.xml:

<role rolename="manager-gui"/>
<role rolename="manager-script"/>
<user username="tomcat" password="tomcat" roles="manager-gui,
 manager-script"/>

modify /usr/local/apache-tomcat-7.0.57/conf/server.xml

change the port to 8888.

 <Connector port="8888" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />

<Connector executor="tomcatThreadPool"
               port="8888" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />

Verify: in /usr/local/apache-tomcat-7.0.57/bin

[root@oc4327053475 bin]# startup.sh
Using CATALINA_BASE:   /usr/local/apache-tomcat-7.0.57
Using CATALINA_HOME:   /usr/local/apache-tomcat-7.0.57
Using CATALINA_TMPDIR: /usr/local/apache-tomcat-7.0.57/temp
Using JRE_HOME:        /usr/java/jre1.8.0_25
Using CLASSPATH:       /usr/local/apache-tomcat-7.0.57/bin/bootstrap.jar:/usr/local/apache-tomcat-7.0.57/bin/tomcat-juli.jar
Tomcat started.

6. Install CF command line:

 Go to https://github.com/cloudfoundry/cli/releases to download the latest version, install the rpm.

Verify: cf -version

Configuration:

First make sure you have pushed all the code to jazzhub or github, in this case, I use jazzhub.

Step 1:

Add necessary plugins. By default, jenkins 1.580.1-1.1 has installed all the necessary plugins(Git, marven, deploy to container), if not, you should install them first from "Manage Jenkins" -> "Manage Plugins" menu.

After that, make sure, if you "New Item" , you should see "Build a maven project" option:

Step 2: Config system:

To config some system level options used in Jenkins. From "Manage Jenkins" -> "Config System"

Add the tools location in Global properties -> Tool Locations

In the Git section, fill the "Path to Git executeable" as "/usr/local/git/bin/git" where we installed git.

Create a new project:

In the Jenkins web UI, click "New Item", create a maven project and named "RestDeploy", then config the project:

1.Git section: input the git repository URL: https://hub.jazz.net/git/wangpbj/RestDeploy with the username/password.

IMPORTANT: Here I have a problem with git 1.7.1, and reportedUnable to find remote helper for 'https' , it proved that, the git version matter, after I changed to git 1.9.4, the problem disappeared. It will automatically connect to the repository, and will report error if has problem.

In the "Build Triggers" section, I select Poll SCM every 15 mins.

In the "Build" section, set "Root POM" as "RestDeploy/pom.xml", "Goals and options" as "clean install", click "advanced" menu, to check "Use private Maven repository", in the "Settings file" select "Settings file in filesystem", and fill the file path "settings.xml".

INPORTANT: By default, my settings file locates in /root/.m2/settings.xml, but it reports can not find the file, so I have to copy the settings.xml file to  /var/lib/jenkins/jobs/RestDeploy/workspace it seems Jenkins search the folder as root folder. Jenkins will use “/var/lib/jenkins/.m2/” as its own maven repository.

Because I want to use mirror to Maven central site, I setup a nexus server locally, it could be much quick, and could hold our own plugins/libs(such as db2 jdbc driver) to shared within teams. the content of the settings.xml:

<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <pluginGroups></pluginGroups>
  <proxies></proxies>
   <servers>     
    <server>  
        <id>nexus-snapshots</id>  
        <username>admin</username>  
        <password>admin123</password>  
    </server>

    <server>  
        <id>tomcat</id>  
        <username>tomcat</username>  
        <password>tomcat</password>  
    </server>  
  </servers>     
  <mirrors>     
    <mirror>  
        <id>nexus</id>  
        <name>internal nexus repository</name>  
        <url>http://9.123.149.131:8081/nexus/content/groups/public/</url>  
        <mirrorOf>central</mirrorOf>  
    </mirror>       
  </mirrors>
  <profiles></profiles>
</settings>

In the post-build actions section, select "Deploy war/ear to a container" , context path: /RestDeploy, select "Tomcat 7.x" as container, and set "tomcat"/"tomcat" as username and password. Tomcat url: http://localhost:8888

How to push to Bluemix:

Step 1:

Create a file "manifest.xml" in  /var/lib/jenkins/jobs/RestDeploy/workspace/RestDeploy/

applications:
- disk_quota: 512M
  host: JavaSQLStony
  name: JavaSQLStony
  path: target/RestDeploy.war
  domain: mybluemix.net
  instances: 1
  memory: 256M

Step 2: Add shell script:

cf login -a https://api.ng.bluemix.net -u wangpbj@cn.ibm.com -p ******
cf push -f RestDeploy/manifest.xml

Now after, bluemix will be up to date after you pushed your code to jazzhub.

Everytime restart the Linux, need to do the following steps:

1.Startup nexus:

cd /root/nexus-2.10.0-02/bin

export RUN_AS_USER=root

./nexus start

2. Startup tomcat:

cd /usr/local/apache-tomcat-7.0.57/bin

startup.sh

3. Open port 8888 for tomcat and 4000 for nomachine for remote control:

cd /sbin

iptables -I INPUT -p tcp --dport 8888 -j ACCEPT

iptables -I OUTPUT -p tcp --sport 8888 -j ACCEPT

iptables -I INPUT -p tcp --dport 4000 -j ACCEPT

iptables -I OUTPUT -p tcp --sport 4000 -j ACCEPT
iptables -I INPUT -p tcp --dport 8080 -j ACCEPT
iptables -I OUTPUT -p tcp --sport 8080 -j ACCEPT
iptables -I INPUT -p tcp --dport 8081 -j ACCEPT
iptables -I OUTPUT -p tcp --sport 8081 -j ACCEPT
iptables -I INPUT -p tcp --dport 9090 -j ACCEPT
iptables -I OUTPUT -p tcp --sport 9090 -j ACCEPT
iptables -I INPUT -p tcp --dport 29418 -j ACCEPT
iptables -I OUTPUT -p tcp --sport 29418 -j ACCEPT

4. Startup jenkins:

service jenkins start

5. Startup ssh:

service sshd start

6. Startup Gerrit:

cd /root/gerrit_test_site/bin

gerrit.sh start

Test：

Everytime you push code to jazzhub: https://hub.jazz.net/git/wangpbj/RestDeploy

After 15 mins(I set the 1 min/poll before, but server is only an out of date desktop machine, it is fully occupied and very slow...), the staging server will show the latest version show:

http://9.123.149.131:8888/RestDeploy

TODOs if I have time:

1. Write a shell script to execute the startup process.

2. Auto push to Bluemix.

3. Add test framework.

4. Add code inspect framework.