Tuesday, 10 May 2016

Parameterised Builds rock!

Welcome to the latest JenkinsHeaven post!

We have a lot to cover in this post. I decided against splitting this material into multiple posts as it logically belongs together and want the reading experience to be as simple as possible.

It may surprise some of my readers but until last week I had not used parameterised builds before.

Well I'm here to let you know that they are awesome and will rock your world, baby!

Today's post is a case of needs must.

Our application at work is growing along a number of axes:

  • To deploy the full solution means deploying two independent systems. In a future release this is expected to grow to three.
  • We are also about to deploy the current release to production with a road-map of at least three future releases.
  • There are three pre-production test environments that Release Candidates can be deployed to before being deployed to production.

All this spells an explosion in the number of jobs making it difficult to manage. This won't do. Parameterised builds are the solution to this problem.

Our updated taxonomy has:

  1. Jobs per independent system
    1. Polling the repository building every commit
    2. Deploying a specified labelled version (Master branch first test env)
  2. Jobs per independent system & release branch
    1. Deploying a specified labelled version to the specified pre-production environment
    2. Deploying a specified labelled version to production

That's it. We now support multiple systems, environments and releases with fewer total jobs than before. Boom! With the CloudBees Folders plugin...Everything. Becomes. Clear! (Tip: If you have an existing job that has builds and is holding a workspace, you might find that you need to Wipeout the Workspace after moving it into a Folder to get it to build correctly again. I had to do this with the TFS plugin).

Note that we have decided that the master branch will always represent latest and greatest and spawn release branches at the time we first deploy to the first of two UAT environments. This allows the team to keep working on the next release and commit UAT bug fixes to the release branch. This approach borrows ideas from the GitFlow approach to branching (BTW: we are using TFS for source control only) and seems to be working well for us.

In that context, let's look at each of these job groups in more detail.

Group 1A - Build every commit

One job per branch (i.e. Master Branch and each release branch) that polls the repository's master branch, runs its unit test suite and static analysis (StyleCop and FxCop). If all the unit tests pass and static analysis is within tolerances, label the repository ({JOBNAME}_{BUILDNUMBER}).

Post Build Notification actions for this group are:

  • XUnit Unit Test publication
  • Code coverage with OpenCover and ReportGenerator
  • Email notification sent to the development and test team which includes unit test results and console log with the Extended Email Plugin. Emails are sent on Failure-Any, Unstable-TestFailures and Success

Group 1B - Deploy master branch

This is the first of our parameterised jobs. It uses two awesome plugins: the excellent Active Choices plugin and the Scriptler plugin (Scriptler is Groovy! Sorry, I couldn't resist.) which work together and will enable you to deploy a labelled version of a branch to an environment with two groovy scripts. We'll look at those scripts later.

The purpose of this group of jobs is to deploy a labelled version (remember a labelled version is one that has passed the unit tests and static analysis in a 1A job) of the master branch to the test team's environment (we simply call it TEST). This is the first test environment after development and is where story validation occurs.

These jobs give development and test team members the ability to deploy to the TEST environment and no further.

Post Build Notification actions for this group are:

  • Email notification same as Group 1A

Group 2A - Deploy Release branches

The remaining two pre-production environments are End to End (E2E) and User Acceptance Test (UAT)

Each job in this group is essentially a copy of Group 1B except it:

  • operates on a release branch
  • the groovy scripts are parameterised to pick up the labels (created by the corresponding 1A job) on the release branch
  • the target environments are E2E and UAT.

In the same way that test team members can "pull" versions through to the TEST environment they are testing in, the E2E and UAT Testers can "pull" versions through to the E2E and UAT environments they are testing in. It is the responsibility of the development team to commit fixes to the appropriate branch.

Post Build Notification actions for this group are:

  • Email notification same as Group 1A

Group 2B - Production Deployments

Each job in this group is essentially a copy of Group 2A except the only available target environment is Production. I did this so that only a small group of people could deploy to production and a wider group could not accidentally deploy to production. At a later date I plan to consolidate Group 2A and 2B by incorporating this (Thanks Bruno!).

Now, those two scripts:

Post Build Notification actions for this group are:

  • Email notification same as Group 1A

GetSuccessfulBuilds

def builds = []

def job = jenkins.model.Jenkins.instance.getItemByFullName($FULL_JOB_NAME)

job.builds.each {
    def build = it
    def label = "L" + $JOB_NAME + "_" + build.displayName[1..-1]
    if (hudson.model.Result.SUCCESS == it.result) {
        builds.add(label)
    }
}

return builds;

The $JOB_NAME parameter is the Jenkins job name (should be self-explanatory) and the $FULL_JOB_NAME includes the CloudBees Folder name like so: <FOLDER_NAME>/<JOB_NAME>. This script needs to set a parameter called VERSION_SPEC. This is so the TFS plugin knows to checkout by label.

GetTargetEnvironments

if($IS_PROD_BUILD.toBoolean()) {
    return ["PROD"]
}
else {
    return ["E2E", "UAT", "PILOT"]
}

I pass the text 'true' or 'false' (without the quotes) for the boolean parameter $IS_PROD_BUILD. This script sets a parameter simply called ENV.

Thanks for reading and hopefully you have found this helpful. As always, if you have any questions, feedback or comments leave them in the comments section below.

Till next time...

2 comments:

  1. Your content is awesome . You have done a great job and its very useful for me . I appreciate your effort and I hope that you will get more positive comments from the web users.
    Seo Web design company Chennai

    ReplyDelete
  2. Thanks for the positive feedback Rekhila!

    ReplyDelete