Pipeline Related Knowledge

Pipeline Step Example

• node(Label Expression){}
  Label Expression: you choose the run VM
  node can include other node

• ws(workspaceName){}
  define your job workspace, path: JENKINS_HOME/workspaceName

• stage(stagename){}
  stagename will display on stafe view

• tool name: ‘BM_ANT’, type: ‘ant’
  use the tool which you setup it on the jenkins configure syatem, such as ant, git

• withEnv([“ANT_HOME=${tool ‘BM_ANT194’}”]){}
  set up environment variable in the running slave, but the variable is process level and its scopes just for code in {}

• bat/sh
  bat: run windows batch script; sh: run shell script

• try{}catch(error){}finally{}
  catch issue

• NUnitPublisher
  step([$class: ‘NUnitPublisher’, testResultsPattern: ‘**/UnitTestResult/*.xml’])
  publisher the NUnit report

• Artifactory

//note : slave java version should be same as srtifactory require
def server = Artifactory.server 'Jfrog_artifactory' //Jfrog_artifactory is defined on jenkins configure system
def patternPath = "${PROJECTNAME}/Installer/${MSIBUILDNAME}${PRODUCT_VERSION}.msi"
def targetPath = "repName/1.0.1/"
def uploadSpec = """{
"files": [
{
"pattern": "${patternPath}",
"target": "${targetPath}"
}
]
}"""
def buildInfo = server.upload(uploadSpec)
def artifactoryPath = server.getUrl()+"/${targetPath}"+"${MSIBUILDNAME}${PRODUCT_VERSION}.msi"

• AzCopy

def AzCopyHome = "\"C:/Program Files (x86)/Microsoft SDKs/Azure/AzCopy/AzCopy.exe\""
def SourcePath = "/Source:\"${env.WORKSPACE}"+"\\${PROJECTNAME}\\Installer\""
def DestPath = "/Dest:\"https://sql****t.blob.core.windows.net/buildinstaller\""
def DestKey = "/DestKey:\"IhUM2xkihQsEfYF9MdqZ********XAGQNkCIKn+WruUMTTxSx5V0u3kBfMUDvf/dPdA==\""
def Pattern = "/Pattern:\"**.msi\""
def AzCopyCommand = "${AzCopyHome} "+"${SourcePath} "+"${DestPath} "+"${DestKey} "+"${Pattern}"
bat AzCopyCommand

• Emailext

def bodyhtml = readFile "**.html"
emailext (
      subject: "your subject title",
      body: bodyhtml,
      recipientProviders: [[$class: 'DevelopersRecipientProvider']],
      to: EmailRECIPIENT
    )

• publishHTML

publishHTML(target: [reportName:"Report Name",
                    reportDir:"Folder contain report",
                    reportFiles:"HTML report .htm")

• build

build job:'../Job path to be triggered',
      wait: false,
      parameters: [[$class: 'StringParameterValue',
                    name: 'BRANCHNAME', value: $branchname],
                   [$class: 'StringParameterValue',
                    name: 'BUILDNUMBER', value: "$buildid"]]
 
 
build job: 'LaunchTE', parameters: [[$class: 'NodeParameterValue', name: 'Label', labels: ['win10TE'],nodeEligibility: [$class: 'AllNodeEligibility']]]
}

• parallel

//parallel create azure VM, ignore build result for going code
    node('bmwin10'){
        parallel (
            task1:{
                stage('test1'){
                    build job: 'keepVM10'
                }
                 
                node('bmwin10'){
                    sleep(30)
                    echo 'task sleep 30 complete!'
                    try{
                           bat "java 10-version"
                       }catch(error){
                           echo "task1:error"
                           //throw error
                           //currentBuild.result = 'FALURE'
                           //currentBuild.result = 'UNSTABLE'
                           currentBuild.result = 'SUCCESS'
                       }
                     
                }
            },
            task2:{
                node('bmwin10'){
                    sleep(20)
                    echo 'task sleep 20 complete!'
                }
            },
            task3:{
                node('bmwin10'){
                    sleep(40)
                    echo 'task sleep 40 complete!'
                }
            }
        )
        stage('test2'){
            build job: 'keepVM10', wait: false
        }  
        sleep(10)
        echo 'task testing end !'
         
    }  
}

• properties

properties([[$class: 'RebuildSettings', autoRebuild: false, rebuildDisabled: false], parameters([string(defaultValue: 'testing', description: 'string parameter testing', name: 'Testing')]), pipelineTriggers([])])
 
 
node {
    echo "${params.Testing} World!"
    def testingDefault = "testingDefault "
    def Testing= "${binding.hasVariable('Testing') ? Testing:testingDefault  }"
}

• Publish Multiple Robot Test Results

node('robot') {
    ...
    publishTestResults('journey')
    publishTestResults('regression')
}
 
void publishTestResults(String type) {
step([
        $class           : 'hudson.plugins.robot.RobotPublisher',
        outputPath       : 'portfolio-app\\target\\robot-output\\' + type,
        passThreshold    : 100,
        unstableThreshold: 100,
        otherFiles       : '',
        reportFileName   : '*\\report*.html',
        logFileName      : '*\\log*.html',
        outputFileName   : '*\\output*.xml'
])

• Read XML file data as map

def readProjectsReferFoldersPropXML(){
    def XMLText = readFile ProjectsReferFoldersPropXMLPath
    def Projects = new XmlSlurper().parseText(XMLText)
    def projectsReferFolders  = [:]
    for(i=0;i<Projects.Project.size();i++){
        def projectName = Projects.Project[i].@name as String
        def folderList=[]
        for(k=0;k<Projects.Project[i].ReferFolders.Folder.size();k++){
            def folderName = Projects.Project[i].ReferFolders.Folder[k] as String
            folderList.add(folderName)
        }
        projectsReferFolders.put(projectName,folderList)
    }
    println projectsReferFolders
    return projectsReferFolders
}
 
/*
XML example:
<?xml version="1.0" encoding="UTF-8"?>
<Projects>
    <Project name="SOEP">
        <ReferFolders>
            <Folder><![CDATA[Build]]></Folder>
            <Folder><![CDATA[Common]]></Folder>
            <Folder><![CDATA[Jenkinsfile]]></Folder>
            <Folder><![CDATA[Execution Plan]]></Folder>
        </ReferFolders>
    </Project>
 
    <Project name="DB2">
        <ReferFolders>
            <Folder><![CDATA[Build]]></Folder>
            <Folder><![CDATA[Common]]></Folder>
            <Folder><![CDATA[Jenkinsfile]]></Folder>
            <Folder><![CDATA[SQLOpt DB2]]></Folder>
        </ReferFolders>
    </Project>
</Projects>
*/

• Pipeline Grammar Point

1. Gstring : " " , ' ',"""
def path = '123456'
'$path' => print $path
"$path" => print 123456
""$path""=> print 123456
2. %% => %
3. \" =>"; "" =>"

• Declarative Pipeline

   https://jenkins.io/doc/book/pipeline/syntax/
  

Simple Pipeline and multibranch pipeline

1. simple pipeline has no the environment variable BRANCH_NAME, but the multibranch has

2. Git push the commit to jenkins server and trigger the multibranch pipeline job, the premise is your git enable a services to allow Jenkins to handle GitHub’s event, just the project administrator has the permission.
   

3. multibranch pipeline should put a file named Jenkinsfile on the project root path.

4. list all environment variable

//windows
bat 'set > env.txt'
//linux
sh 'env > env.txt'
 
for (String i : readFile('env.txt').split("\r?\n")) {
    println i
}

After one stage failed, should stop other stages, add the check like this, if there just CurrentBuild.result = ‘Aborted’, it still run next stage, need the error(’’) code to stop.

script{
       status = powershell(returnStatus: true, script: 'JenkinsLib/UpdateBuildNumAndCopyDoc.ps1')
                   println status
             if (status!=0){
                currentBuild.result = 'ABORTED'
                      error('Error Stopping with Prepare step')
             }

Reference Link

• Top 10 Best Practices for Jenkins Pipeline Plugin
  https://www.cloudbees.com/blog/top-10-best-practices-jenkins-pipeline-plugin

• Pipeline Syntax
  https://jenkins.io/doc/book/pipeline/syntax/

• Getting Started
  https://jenkins.io/doc/pipeline/tour/hello-world/

• Parallelism and Distributed Builds with Jenkins
  https://www.cloudbees.com/blog/parallelism-and-distributed-builds-jenkins

• Declarative Pipeline
  https://jenkins.io/doc/book/pipeline/syntax/

Some useful link:
https://www.linkedin.com/pulse/new-best-practices-jenkins-pipeline-global-shared-libraries-rossi/
https://jenkins.io/blog/2017/10/02/pipeline-templates-with-shared-libraries/
https://jenkins.io/doc/book/pipeline/shared-libraries/