book
  • README
  • cheatsheet
    • bash
      • builtin
      • syntactic sugar
      • cmd
      • havefun
    • text-processing
      • awk
      • sed
      • html
      • json
      • regex
      • unicode
    • osx
    • curl
    • tricky
    • widget
    • proxy
    • colors
    • math
    • media
    • ssl
      • keystore
      • verification
      • server
      • client
      • tricky
    • windows
      • powershell
      • choco
      • wsl
      • wt
      • shortcut
      • clsid
      • env
      • shell:folder
  • vim
    • nvim
    • install
    • color
    • plugins
      • usage
      • other plugins
      • deprecated
    • tricky
    • viml
    • windows
    • troubleshooting
  • devops
    • admin tools
    • ssh
    • git
      • config
      • alias
      • submodule
      • eol
      • example
      • gerrit
        • gerrit API
      • github
      • troubleshooting
      • tricky
      • statistics
    • pre-commit
    • release-tools
    • tmux
      • cheatsheet
    • ansible
    • vault
    • artifactory
      • api
      • cli
      • aql
      • nginx cert
    • klocwork
      • kwadmin
      • kwserver
      • api
      • q&a
    • elk
    • mongodb
    • android
    • mobile
  • jenkins
    • config
      • windows
    • appearance
    • troubleshooting
    • jenkinsfile
      • utility
      • parallel
      • build
      • envvar
      • properties
      • trigger
      • node
    • script
      • job
      • build
      • stage
      • agent
      • security & authorization
      • exception
      • monitor
      • tricky
    • api
      • blueocean
    • cli
    • plugins
      • kubernetes
      • docker
      • shared-libs
      • lockable-resource
      • ansicolor
      • badge
      • groovy-postbuild
      • simple-theme
      • customizable-header
      • artifactory
      • jira-steps
      • job-dsl
      • build-timeline
      • crumbIssuer
      • coverage
      • uno-choice
      • tricky
  • virtualization
    • kubernetes
      • init
        • kubespray
        • kubeadm
          • environment
          • crio v1.30.4
          • docker v1.15.3
          • HA
        • addons
        • etcd
      • kubectl
        • pod
        • deploy
        • replicasets
        • namespace
        • secrets
      • node
      • certificates
      • events
      • kubeconfig
      • kubelet
      • troubleshooting
      • cheatsheet
      • auth
      • api
      • tools
        • monitor
        • helm
        • network
        • minikube
    • docker
      • run & exec
      • voume
      • remove
      • show info
      • dockerfile
      • dockerd
      • tricky
      • troubleshooting
      • windows
    • crio
    • podman
  • ai
    • prompt
  • osx
    • apps
      • init
      • brew
    • defaults
    • system
    • network
    • script
    • tricky
  • linux
    • devenv
    • util
      • time & date
      • output formatting
      • params
      • tricky
    • nutshell
    • disk
    • network
    • troubleshooting
    • system
      • apt/yum/snap
      • authorization
      • apps
      • x11
    • ubuntu
      • systemctl
      • x
    • rpi
  • programming
    • groovy
    • python
      • config
      • basic
      • list
      • pip
      • q&a
    • others
    • archive
      • angular
      • maven
      • mysql
        • installation
        • logs
      • ruby
        • rubyInstallationQ&A
  • tools
    • fonts
    • html & css
    • Jira & Confluence
    • node & npm
      • gitbook
      • hexo
      • github.page
      • code themes
    • app
      • microsoft office
      • vscode
      • virtualbox
      • iterm2
      • browser
      • skype
      • teamviewer
      • others
  • quotes
  • english
Powered by GitBook
On this page
  • pipeline utility steps
  • DSL
  • others
  • withCredentials
  • code clone
  • groovy.io.FileType

Was this helpful?

  1. jenkins
  2. jenkinsfile

utility

PreviousjenkinsfileNextparallel

Last updated 10 months ago

Was this helpful?

findFiles

  • jenkinsfile

    sh "touch a.txt"
    def files = findFiles ( glob: "**/*.txt" )
    println """
              name : ${files[0].name}
              path : ${files[0].path}
         directory : ${files[0].directory}
            length : ${files[0].length}
      lastModified : ${files[0].lastModified}
    """
  • result

    [Pipeline] sh (hide)
    + touch a.txt
    [Pipeline] findFiles
    [Pipeline] echo
    
                  name : a.txt
                  path : a.txt
             directory : false
                length : 0
          lastModified : 1605525397000
try {
  sh 'might fail'
  mail subject: 'all well', to: 'admin@somewhere', body: 'All well.'
} catch (e) {
  def w = new StringWriter()
  e.printStackTrace(new PrintWriter(w))
  mail subject: "failed with ${e.message}", to: 'admin@somewhere', body: "Failed: ${w}"
  throw e
}

tips

java.io.NotSerializableException: groovy.lang.IntRange

refenrece:

  • caused via

    (1..5).each { println it }
  • solution

    • (1..5).flatten().each { println it }
    • (1..5).toList().each { println it }

references:

load a constant

groovy file ( constant.groovy )

@Field final Map<String, Map<String, String>> FOO = [
  'k1' : [ 'k11' : 'v11' ],
  'k2' : [ 'k21' : 'v21', 'k22' : 'v22' ]
]

Jenkinsfile

Object matrix = load( '/path/to/constant.groovy' )
if ( matrix.metaClass.hasProperty(matrix, 'FOO') ) {
  println """
       matrix.FOO : ${matrix.FOO}
    matrix.FOO.k1 : ${matrix.FOO.k1}
      matrix.type : ${matrix.metaClass.hasProperty(matrix, 'FOO').type}
  """
}
  • result

         matrix.FOO : [k1:[k11:v11], k2:[k21:v21, k22:v22]]
      matrix.FOO.k1 : [k11:v11]
        matrix.type : interface java.util.Map

extend the pipeline

groovy script ( builtInStage.groovy )

String str = "this is string A"

def stageA( String path ) {
  stage('scenario A') { println path }
}

def stageB() {
  stage('scenario B') { println str }
}

def stageC() {
  stage('scenario C') {
    String strC = "this is string C"
    sh "echo ${strC}"
    println strC
  }
}

def runAllStages( String basePath ) {
  stageA( basePath )
  stageB()
  stageC()
}

return this

jenkinsfile

node('controller') {
  stage( 'download code' ) {
    echo 'prepare'
    sh "cat ${WORKSPACE}/builtInStage.groovy"
  }

  test = load "builtInStage.groovy"
  test.runAllStages( "${WORKSPACE}/mypath" )

  stage( 'publish' ) {
    echo 'done!'
  }
}

timeout

references:

sample jenkins pipeline

import jenkins.model.CauseOfInterruption
import org.jenkinsci.plugins.workflow.steps.TimeoutStepExecution

stage( 'wating' ) {
  catchError( buildResult: 'SUCCESS', stageResult: 'ABORTED' ) {

    try {

      timeout( time: 5, unit: 'SECONDS' ) { sleep( time: 10, unit: 'SECONDS' ) }

    } catch ( org.jenkinsci.plugins.workflow.steps.FlowInterruptedException e ) {

      String msg
      if ( e.causes[0] instanceof TimeoutStepExecution.ExceededTimeout ) {
        msg = 'NOT_BUILT: aborted by timeout'
        currentBuild.result = 'NOT_BUILT'
      } else if ( e.causes[0] instanceof CauseOfInterruption.UserInterruption ) {
        User user = e.causes[0]?.user
        println "${user.fullName} : ${user.absoluteUrl}"
        msg = "ABORTED : by user : ${user.displayName} [ ${user.id} ]"
        currentBuild.result = 'ABORTED'
      }
      println "${msg}"
      currentBuild.description = msg

    } catch ( Throwable e ) {

      def sw = new StringWriter()
      e.printStackTrace(new PrintWriter(sw))
      println sw.toString()
      throw e

    } // try | catch

  } // catchError
} // stage

get previous build abort

[!TIP]

// jenksinfile
println libs.isTimeout( env.JOB_NAME, currentBuild.previousBuild.number )

// libs.groovy
Boolean isTimeout( String name, int nubmer ) {
  Jenkins.instance
         .getItemByFullName( name )
         .getBuildByNumber( nubmer )
         .getActions( jenkins.model.InterruptedBuildAction.class )
         .causes
         .flatten()
         .any{ it instanceof TimeoutStepExecution.ExceededTimeout }
}

DSL

original DSL:

cleanWs(
  deleteDirs: true,
  disableDeferredWipeout: true,
  notFailBuild: true,
  patterns: [
    [ pattern: 'b.txt', type: 'INCLUDE' ] ,
    [ pattern: 'a.txt', type: 'INCLUDE' ]
  ]
)
  • List p = [ 'a.txt', 'b.txt' ]
    
    cleanWs(
      deleteDirs: true,
      disableDeferredWipeout: true,
      notFailBuild: true,
      patterns: [
        *p.collect {
          [pattern: "${it}", type: 'INCLUDE']
        }
      ]
    )
  • List.collect

    List p = [ 'a.txt', 'b.txt' ]
    
    cleanWs(
      deleteDirs: true,
      disableDeferredWipeout: true,
      notFailBuild: true,
      patterns: p.collect { [pattern: "${it}", type: 'INCLUDE'] }
    )
  • with API

    import hudson.plugins.ws_cleanup.Pattern
    import hudson.plugins.ws_cleanup.Pattern.PatternType
    
    List p = [ 'a.txt', 'b.txt' ]
    
    cleanWs(
      deleteDirs: true,
      disableDeferredWipeout: true,
      notFailBuild: true,
      patterns: p.collect { new Pattern(it, PatternType.INCLUDE) }
    )
  • with pure API

    Javadoc:

    get FilePath :

    import hudson.plugins.ws_cleanup.WsCleanup
    import hudson.plugins.ws_cleanup.Pattern
    import hudson.plugins.ws_cleanup.Pattern.PatternType
    
    List p = [ 'a.txt', 'b.txt' ]
    
    WsCleanup wsc = new WsCleanup()
    wsc.setDeleteDirs(true)
    wsc.setDisableDeferredWipeout(true)
    wsc.setNotFailBuild(true)
    wsc.setPatterns(
      p.each {
        new Pattern( it, PatternType.INCLUDE )
      }
    )
    
    // unresolved
    wsc.perform( currentBuild.rawBuild, <FilePath>, <Launcher>, <TaskListener> )

others

handle api

import groovy.json.JsonSlurper

String username = System.getenv('GITHUB_USERNAME')
String password = System.getenv('GITHUB_PASSWORD')

String GITHUB_API = 'https://api.github.com/repos'
String repo = 'groovy'
String PR_ID = '2' // Pull request ID

String url = "${GITHUB_API}/${username}/${repo}/pulls/${PR_ID}"
println "Querying ${url}"
def text = url.toURL().getText(requestProperties: ['Authorization': "token ${password}"])
def json = new JsonSlurper().parseText(text)
def bodyText = json.body

// Check if Pull Request body has certain text
if ( bodyText.find('Safari') ) {
    println 'Found Safari user'
}

running in temporaray folders

  • node ( 'built-in' ) {
      dir( pwd('tmp':true) ) {
        sh """
          pwd
          ls -altrh
        """
      } //dir
    } // node
    • console output

      16:13:46  + pwd
      16:13:46  /home/devops/workspace/marslo/sandbox/func@tmp
      16:13:46  + ls -altrh
      16:13:46  total 0
      16:13:46  drwxr-xr-x 4 devops devops 42 Jul 13 08:13 ..
      16:13:46  drwxr-xr-x 3 devops devops 19 Jul 13 08:13 .
  • using System.currentTimeMillis()

    node ( 'built-in' ) {
      dir( System.currentTimeMillis().toString() ) {
        sh """
          pwd
          ls -altrh
        """
      } //dir
    } // node
    • console output

      16:26:14  + pwd
      16:26:14  /home/devops/workspace/marslo/sandbox/func/1657700773771
      16:26:14  + ls -altrh
      16:26:14  total 0
      16:26:14  drwxr-xr-x 2 devops devops  6 Jul 13 08:26 .
      16:26:14  drwxr-xr-x 4 devops devops 52 Jul 13 08:26 ..

check git branch exists in local repo

/**
 * check whether if the git refs exists in local repo or not
 *
 * @param name         the git base references, can be branch name or revision or tag or pointer refs ( i.e. {@code HEAD}, {@code FETCH_HEAD}, ... )
 * @param type         the references type. can be {@code [ heads | tags | refs ]}. revision will be considered as {@code refs}
 * @param dir          the local repo location. using current directory by default
 * @param verbose      whether or not to show debug information
**/
Boolean hasLocalReference( String name          ,
                           String type = 'refs' ,
                           String dir  = pwd()  ,
                           Boolean verbose = true
) {
  if ( ! [ 'heads', 'tags', 'refs' ].contains(type) ) println( "ERROR: invalid type ! available type : 'heads', 'tags' or 'refs'" )
  if ( verbose ) println ( "~~> check whether if ${type} : '${name}' exists in local repository : '${dir}'" )
  String refs = [ 'heads', 'tags' ].contains(type) ? "refs/${type}/${name}" : name
  sh ( returnStatus : true ,
       script : """
         set -x ;
         [ -d "${dir}" ] && \
         git -C "${dir}" rev-parse --is-inside-work-tree >/dev/null 2>&1 && \
         git -C "${dir}" cat-file -e ${refs}
       """
  ) == 0
}
  • or via show-ref

    /**
     * check whether if the branch exists in local repository
     *
     * @param name         the branch name will be checked
     * @param type         to using {@code refs/${type}} to check branch or tags in local
     * @param dir          the local repo location. using current directory by default
     * @param verbose      whether or not to show debug information
    **/
    Boolean hasLocalBranch ( String name        ,
                             String type        ,
                             String dir = pwd() ,
                             Boolean verbose = true
    ) {
      if ( ! [ 'heads', 'tags' ].contains(type) ) util.showError( "ERROR: invalid type! available type : 'heads' or 'tags'" )
      if ( verbose ) color.echo( LOGGER_COLOR, "~~> check whether if ${refName(type)} : '${name}' exists in local repository : '${dir}'" )
      sh ( returnStatus : true ,
           script : """
             set +x;
             [ -d "${dir}" ] && \
             git -C "${dir}" rev-parse --is-inside-work-tree >/dev/null 2>&1 && \
             git -C '${dir}' show-ref --quiet refs/${type}/${name}
           """
      ) == 0
    } // hasLocalBranch

withCredentials

references:

Both `gitUsernamePassword` and `gitSshPrivateKey` bindings depend on the Credential Plugin to retrieve user’s credential using the Credentials API. Git SSH Private Key Binding The gitSshPrivateKey implementation provides git authentication support over SSH protocol using private key and passphrase credentials of a user. The binding uses two git specific environment variables depending upon the minimum CLI-git version - `GIT_SSH_COMMAND` - If version is greater than or equal to 2.3, then the GIT_SSH_COMMAND environment variable provides the ssh command including necessary options which are: path to the private key and host key checking, to authenticate and connect to the git server without using an executable script. - `SSH_ASKPASS` - If version is less than 2.3, an executable script is attached to the variable which provides the ssh command including necessary options which are: path to the private key and host key checking, to authenticate and connect to the git server

push with ssh private credentials

references:

for username & password by gitUsernamePassword :

withCredentials([
  gitUsernamePassword( credentialsId: 'my-credentials-id', gitToolName: 'git-tool' )
]) {
  bat 'git submodule update --init --recursive'
}

[!NOTE] If for any particular reason, the push must be done using a different method the URL needs to be configured accordingly:

  • git config url.git@github.com/.insteadOf https://github.com/ : if the checkout was done through HTTPS but push must be done using SSH

  • git config url.https://github.com/.insteadOf git@github.com/ : if the checkout was done through SSH but push must be done using HTTPS escaping characters in the ECHO commands of the askpass script:

.replace("%", "%%")
.replace("^", "^^")
.replace("&", "^&")
.replace("<", "^<")
.replace(">", "^>")
.replace("|", "^|")
.replace("'", "^'")
.replace("`", "^`")
.replace(",", "^,")
.replace(";", "^;")
.replace("=", "^=")
.replace("(", "^(")
.replace(")", "^)")
.replace("!", "^!")
.replace("\"", "^\"")
withCredentials([ sshUserPrivateKey(
                       credentialsId : 'GITSSHPRIVATEKEY',
                     keyFileVariable : 'SSHKEY',
                    usernameVariable : 'USERNAME'
                  )
]) {
  sh """
    GIT_SSH_COMMAND="ssh -i ${SSHKEY} -o User=${USERNAME} -o StrictHostKeyChecking=no" \
    git ls-remote <repoUrl> --heads \
    git push origin <local-branch>:<remote-branch>
  """
}
  • or

    def withSSHCredential( String credential, Boolean verbose = true ) {
        [
          run : { command ->
            if ( verbose ) println ( "~~> run '${command}' with credential ${credential} :" )
            withCredentials([ sshUserPrivateKey(
                                   credentialsId : credentialsId ,
                                 keyFileVariable : 'SSHKEY'      ,
                                usernameVariable : 'USERNAME'
                              )
            ]) {
              String sshCommand = "GIT_SSH_COMMAND=\"ssh -i '${SSHKEY}'   " +
                                                        "-l '${USERNAME}' " +
                                                        "-o StrictHostKeyChecking=no" +
                                                  '"'
              sh ( returnStatus : true ,
                         script : """ set +x -e ; ${sshCommand} ${command} """
              ) == 0
            } // sshUserPrivateKey
          } // run
        ]
    } // withCredential
    
    // withSSHCredential( 'credentialID' ).run( 'git ls-remote git@github.com:sample/repo.git' )

ssh-agent(https://plugins.jenkins.io/ssh-agent)

references:

  • sample code

    steps {
      sshagent (credentials: ['jenkins-generated-ssh-key']) {
        sh("""
          git tag ${props['DATE_TAG']}
          git push --tags
        """)
      }
    }

code clone

String repo     = 'https://github.com/sample-code'
String branch   = 'develop'
String refsepc  = "+refs/heads/${branch}:/remotes/origin/${branch}"
String revision = 'abcdefg' // or 'HEAD'

try {
  checkout([
    $class: 'GitSCM',
    branches: [[ name: revision ]],
    extensions: [
                  [
                           $class  : 'CloneOption',
                     honorRefspec  : true,
                           noTags  : false,
                        reference  : '',
                          shallow  : false
                  ],
                  [
                            $class : 'AuthorInChangelog'
                  ]
    ],
    userRemoteConfigs: [[
      credentialsId  : <credential_id> ,
            refspec  : branch ,
                url  : repo
    ]]
  ])
} catch ( Exception e ) {
  def sw = new StringWriter()
  e.printStackTrace( new PrintWriter(sw) )
  echo sw.toString()

  if ( e instanceof java.lang.NullPointerException ) {
    error ( "${repo} CANNOT be found !")
  } else if ( e instanceof hudson.AbortException ) {
    println ( "failed to clone repo ${repo}\n${e.message}" )
    if ( 'determineRevisionToBuild' == e.stackTrace.collect { it.methodName }.first() ) {
      error ( "'${revision}' cannot be found in refs : '${branch}' ! provide correct revision number or branch name." + "<br>${e.message}" )
    }
    if ( 'retrieveChanges' == e.stackTrace.collect { it.methodName }.first() ) {
      error ( "'${branch}' or '${revision}' cannot be found in ${repo} !<br>${e.message}" )
    }
  }

  throw e
}

groovy.io.FileType

[!NOTE|label:references:]

  • import static groovy.io.FileType.FILES
import groovy.io.FileType
import static groovy.io.FileType.*

/**
 * Traverse the files or directories in the given path
 *
 * @param path      the path to be traversed
 * @param filetype  the type of files to be traversed. can be {@code [ files | directories | any ]}
 * @param depth     the depth of the traversal. default is 1
 * @return          the list of files or directories in the given path
**/
@NonCPS
def traverseInPath( String path, String filetype, Integer depth = 1 ) {
  List<String> names = []
  if ( ! [ 'files', 'directories', 'any' ].contains(filetype) ) {
      currentBuild.description = "`filetype` support only ${[ 'files', 'directories', 'any' ].join(',')} !"
      currentBuild.result = 'NOT_BUILT'
      currentBuild.getRawBuild().getExecutor().interrupt(Result.NOT_BUILT)
  }

  Closure sortByTypeThenName = { a, b ->
    a.isFile() != b.isFile() ? a.isFile() <=> b.isFile() : a.name.toLowerCase() <=> b.name.toLowerCase()
  }
  new File(path).traverse(
    type     : FileType.valueOf( filetype.toUpperCase() ),
    maxDepth : depth,
    sort     : sortByTypeThenName
  ) {
    names << it
  }
  return names
}
  • way to call

    String path = '/path/to/folder'
    
    println ( ">> traverse all FILES/DIRECTORIES in ${path} in maxDepth 0 : " )
    println traverseInPath( path, 'any', 0 ).join('\n')
    
    println ( ">> traverse all FILES in ${path} in maxDepth 2 : " )
    println traverseInPath( path, 'files', 2 ).join('\n')

see also

stage view

see also

using

|

pipeline utility steps
send mail with catch error
Groovy Range Examples
Groovy For Loop Examples
IntRange.flatten()
IntRange.toList()
Evaluate a Groovy source file into the Pipeline script
Load From File
imarslo: groovy metaClass
plain-catch-blocks
How to time out Jenkins Pipeline stage and keep the pipeline running?
Class TimeoutStepExecution
Class org.jenkinsci.plugins.workflow.steps.TimeoutStepExecution.ExceededTimeout
Class CauseOfInterruption.UserInterruption
Class User
TimeoutStepExecution.java
iMarslo : get builds abort cause
Spread Operator
Groovy Goodness: the Spread Operato
hudson.plugins.ws_cleanup.WsCleanup
hudson.plugins.ws_cleanup.Pattern
hudson.plugins.ws_cleanup.Pattern.PatternType
FilePath
Launcher
TaskListener
Using FilePath to access workspace on slave in Jenkins pipeline
Demo: Processing Github JSON from Groovy
pwd(temp:true)
JENKINS-47514 : Special characters in password are not escaped properly in git plugin's withCredentials
Git Credentials Binding for sh, bat, powershell
Git credentials binding for sh, bat, and powershell
Is it possible to Git merge / push using Jenkins pipeline
Pipeline - Equivalent to Git Publisher
Git from Jenkins pipeline is using wrong SSH private key to push back into Git repository
Credentials Binding Plugin
How is your Jenkins 'controller' configured for SSH push in the Jenkinsfile? #5
Use ssh credentials in jenkins pipeline with ssh, scp or sftp
Groovy Goodness: Traversing a Directory
Groovy Goodness: Traversing a Directory
Recursive listing of all files matching a certain filetype in Groovy
Get a list of all the files in a directory (recursive)
pipeline utility steps
findFiles
send mail with catch error
tips
Evaluate a Groovy source file into the Pipeline script
load a constant
extend the pipeline
timeout
DSL
others
handle api
running in temporaray folders
check git branch exists in local repo
withCredentials
push with ssh private credentials
ssh-agent(https://plugins.jenkins.io/ssh-agent)
code clone
groovy.io.FileType