Jenkins Configuration(3)Shell Script
Jenkins, we can make SVN, GIT, MAVEN, ANT working on that. But we need to restart JBOSS server, copy the package to working directory, remote access and sync files to other servers. So we need the help from Shell Script.
1. Basic Shell script and basic expect script
Here are the shell script, server-deploy.sh
#!/bin/sh
# copy the file to push directory
STORE_DIST=/android/work_space/push/projectname.war
rm -fr $STORE_DIST;
BUILD_TARGET=war
BUILD_WAR=$(ls $BUILD_TARGET/projectname-*-SNAPSHOT.war)
TMP=${BUILD_WAR%%.war}
unzip -q -d ${TMP} "$BUILD_WAR" ;
cp -r ${TMP} $STORE_DIST;
./server-autopush.exp
We need a exp file to remote access other server, because server-deploy.sh is running on the jenkins build server. We need to sync or rather say push the changes to our testing server, production server. Just note them as some other linux machine, and we need to remotely restart their JBOSS.
#!/usr/bin/expect -f
# December 26 2012
# This script can automatically push the files
# to /opt/jboss-as/server/default/deploy/ on testing server.
# TIMEOUT
set timeout 20
# Login parameters
set server testing.server.com
set port 22
set user deploy
set rootpasswd 111111
# Logfile
log_file server-autopush-run.log
## Enable this and Disable the "spawn ssh ..." above if you are using ssh-key.
#spawn ssh -p $port $user@$server
spawn ssh -p $port -i /home/build/.ssh/id_rsa $user@$server
# Login as Root
expect "*]$\ " {send "su - root\r"}
expect "*assword:\ " {send "$rootpasswd\r"}
# Kill Jboss on testing server
expect "*]#\ " {send "kill -9 \`ps aux | grep /opt/jboss-as/bin/run.jar | grep -v grep | awk \'{print \$2}\'\`\r"}
# Sync the files from testing server to
expect "*]#\ " {send "rsync -arz --delete --force -e \"ssh -i /home/rsync/.ssh/rsync_servers.ppk\" rsync@server.com:/android/work_space/push /opt/build/buildserver/\r"}
# Copy the files from /opt/build/buildserver/push to /opt/jboss-as/server/default/deploy on testing server
expect "*]#\ " {send "\\cp -rpa /opt/build/buildserver/push/* /opt/jboss-as/server/default/deploy/\r"}
# Remove direcotries data,log,work,tmp
expect "*]#\ " {send "\\rm -rf /opt/jboss-as/server/default/tmp\r"}
expect "*]#\ " {send "\\rm -rf /opt/jboss-as/server/default/work\r"}
expect "*]#\ " {send "\\rm -rf /opt/jboss-as/server/default/data\r"}
# Start Jboss on testing server
expect "*]#\ " {send "nohup /etc/init.d/jboss start > /tmp/nohup.out \r"}
# Exit Root
expect "*]#\ " {send "exit\r"}
# Exit User
expect "*]$\ " {send "exit\r"}
# Exit Expect
expect eof
But when I run this method on my server, I got these error messages
Error Message:
>log_file: command not found
>spawn: command not found
>expect: command not found
>expect: command not found
>expect: command not found
>expect: command not found
>expect: command not found
>expect: command not found
That is only because I did not have expect on my machine.
Solution:
>yum update
>yum install tcl
>yum install expect
2. Improvement on SVN checkout process
One import improvement for the workspace working with SVN:
Repository URL: babababababalalalalalaal
Local module directory(optional) This is usually be ./
But I my project is a sub project, so I need some core codes standing there. So I make them svn update to one place
/opt/work_space
/opt/jenkins
So I change the ./ to mostly like this:
./../../../../work_space/branches/development/project/sub_projects/myproject
3. Take some parameters in shell script and expect script
3.1 Shell parameters
I made a test shell file like this test.sh
!/bin/sh
echo There are $# arguments to $0: $*
echo first argument: $1
echo second argument: $2
echo third argument: $3
echo fourth argument: $4
echo here they are again: $@
When I execute it with parameters
>./test.sh 1 2 3 4
There are 4 arguments to ./test.sh: 1 2 3 4
first argument: 1
second argument: 2
third argument: 3
fourth argument: 4
here they are again: 1 2 3 4
3.2 Expect parameters
And example in test.exp working with test.sh
append this line in test.sh
./test.exp h1 h2
test.exp content will be like this>
#!/usr/bin/expect -f
set arg1 [lindex $argv 0]
set arg2 [lindex $argv 1]
send $arg1\n
send $arg2\n
The result will be like this:
>./test.sh 1 2 3 4
There are 4 arguments to ./test.sh: 1 2 3 4
first argument: 1
second argument: 2
third argument: 3
fourth argument: 4
here they are again: 1 2 3 4
h1
h2
4. Some Tips about private/public key
Add one user named deploy
>/usr/sbin/useradd deploy
Give the user an password
>passwd deploy
Generate the key pair
>ssh-keygen -t rsa
Add the public key to authorize
>cd ~/.ssh
>vi authorized_keys
>chmod 711 ~/.ssh
>chmod 644 ~/.ssh/authorized_keys
Client Server tries to login to the remote server
>ssh -i /home/build/.ssh/id_rsa deploy@remote.server.com
Give the rights of directory to one user
>chown -R rsync:rsync /directory/name
rsync is the username, the second rsync is the group name.
Check the public key from private key
>ssh-keygen -f id_rsa -y
references:
http://s3tools.org/s3cmd
http://www.garron.me/mac/install-s3cmd-mac-os-x.html
http://wangyan.org/blog/s3cmd-how-to-use.html
http://osr507doc.sco.com/en/OSUserG/_Passing_to_shell_script.html
Jenkins, we can make SVN, GIT, MAVEN, ANT working on that. But we need to restart JBOSS server, copy the package to working directory, remote access and sync files to other servers. So we need the help from Shell Script.
1. Basic Shell script and basic expect script
Here are the shell script, server-deploy.sh
#!/bin/sh
# copy the file to push directory
STORE_DIST=/android/work_space/push/projectname.war
rm -fr $STORE_DIST;
BUILD_TARGET=war
BUILD_WAR=$(ls $BUILD_TARGET/projectname-*-SNAPSHOT.war)
TMP=${BUILD_WAR%%.war}
unzip -q -d ${TMP} "$BUILD_WAR" ;
cp -r ${TMP} $STORE_DIST;
./server-autopush.exp
We need a exp file to remote access other server, because server-deploy.sh is running on the jenkins build server. We need to sync or rather say push the changes to our testing server, production server. Just note them as some other linux machine, and we need to remotely restart their JBOSS.
#!/usr/bin/expect -f
# December 26 2012
# This script can automatically push the files
# to /opt/jboss-as/server/default/deploy/ on testing server.
# TIMEOUT
set timeout 20
# Login parameters
set server testing.server.com
set port 22
set user deploy
set rootpasswd 111111
# Logfile
log_file server-autopush-run.log
## Enable this and Disable the "spawn ssh ..." above if you are using ssh-key.
#spawn ssh -p $port $user@$server
spawn ssh -p $port -i /home/build/.ssh/id_rsa $user@$server
# Login as Root
expect "*]$\ " {send "su - root\r"}
expect "*assword:\ " {send "$rootpasswd\r"}
# Kill Jboss on testing server
expect "*]#\ " {send "kill -9 \`ps aux | grep /opt/jboss-as/bin/run.jar | grep -v grep | awk \'{print \$2}\'\`\r"}
# Sync the files from testing server to
expect "*]#\ " {send "rsync -arz --delete --force -e \"ssh -i /home/rsync/.ssh/rsync_servers.ppk\" rsync@server.com:/android/work_space/push /opt/build/buildserver/\r"}
# Copy the files from /opt/build/buildserver/push to /opt/jboss-as/server/default/deploy on testing server
expect "*]#\ " {send "\\cp -rpa /opt/build/buildserver/push/* /opt/jboss-as/server/default/deploy/\r"}
# Remove direcotries data,log,work,tmp
expect "*]#\ " {send "\\rm -rf /opt/jboss-as/server/default/tmp\r"}
expect "*]#\ " {send "\\rm -rf /opt/jboss-as/server/default/work\r"}
expect "*]#\ " {send "\\rm -rf /opt/jboss-as/server/default/data\r"}
# Start Jboss on testing server
expect "*]#\ " {send "nohup /etc/init.d/jboss start > /tmp/nohup.out \r"}
# Exit Root
expect "*]#\ " {send "exit\r"}
# Exit User
expect "*]$\ " {send "exit\r"}
# Exit Expect
expect eof
But when I run this method on my server, I got these error messages
Error Message:
>log_file: command not found
>spawn: command not found
>expect: command not found
>expect: command not found
>expect: command not found
>expect: command not found
>expect: command not found
>expect: command not found
That is only because I did not have expect on my machine.
Solution:
>yum update
>yum install tcl
>yum install expect
2. Improvement on SVN checkout process
One import improvement for the workspace working with SVN:
Repository URL: babababababalalalalalaal
Local module directory(optional) This is usually be ./
But I my project is a sub project, so I need some core codes standing there. So I make them svn update to one place
/opt/work_space
/opt/jenkins
So I change the ./ to mostly like this:
./../../../../work_space/branches/development/project/sub_projects/myproject
3. Take some parameters in shell script and expect script
3.1 Shell parameters
I made a test shell file like this test.sh
!/bin/sh
echo There are $# arguments to $0: $*
echo first argument: $1
echo second argument: $2
echo third argument: $3
echo fourth argument: $4
echo here they are again: $@
When I execute it with parameters
>./test.sh 1 2 3 4
There are 4 arguments to ./test.sh: 1 2 3 4
first argument: 1
second argument: 2
third argument: 3
fourth argument: 4
here they are again: 1 2 3 4
3.2 Expect parameters
And example in test.exp working with test.sh
append this line in test.sh
./test.exp h1 h2
test.exp content will be like this>
#!/usr/bin/expect -f
set arg1 [lindex $argv 0]
set arg2 [lindex $argv 1]
send $arg1\n
send $arg2\n
The result will be like this:
>./test.sh 1 2 3 4
There are 4 arguments to ./test.sh: 1 2 3 4
first argument: 1
second argument: 2
third argument: 3
fourth argument: 4
here they are again: 1 2 3 4
h1
h2
4. Some Tips about private/public key
Add one user named deploy
>/usr/sbin/useradd deploy
Give the user an password
>passwd deploy
Generate the key pair
>ssh-keygen -t rsa
Add the public key to authorize
>cd ~/.ssh
>vi authorized_keys
>chmod 711 ~/.ssh
>chmod 644 ~/.ssh/authorized_keys
Client Server tries to login to the remote server
>ssh -i /home/build/.ssh/id_rsa deploy@remote.server.com
Give the rights of directory to one user
>chown -R rsync:rsync /directory/name
rsync is the username, the second rsync is the group name.
Check the public key from private key
>ssh-keygen -f id_rsa -y
references:
http://s3tools.org/s3cmd
http://www.garron.me/mac/install-s3cmd-mac-os-x.html
http://wangyan.org/blog/s3cmd-how-to-use.html
http://osr507doc.sco.com/en/OSUserG/_Passing_to_shell_script.html