Set up a continuous delivery framework with Jenkins
The setup of the framework in a continuous delivery process is important. The framework determines DevOps's efficiency and what can be done in the continuous delivery process.
This article contains information on Jenkins and demonstrates how to:
- Set up the continuous delivery framework with Jenkins.
- Apply this knowledge to the continuous delivery framework.
- Implement the continuous delivery framework with Jenkins.
The intended audience for this article are software engineers who work on continuous delivery or continuous automation testing. To follow the steps in this article, you should understand:
- Scripting development.
- The software development process.
Jenkins is a continuous integration tool used most often for software development. This automation framework runs repeated jobs. Jenkins can manage and monitor the start of commands on remote systems and can also execute anything that can run via a command line. Jenkins integrates email, TestNG and other tools with supporting plugins.
After installation (see sidebar if you haven't installed Jenkins yet),
access Jenkins via your browser at
Jenkins supports the master/slave mode. The workload of building projects are delegated to multiple slave nodes. This allows a single Jenkins installation to host a large number of projects, or to provide different environments needed for builds/tests.
Set up and enable Jenkins
Before you can use Jenkins, you need to configure it. In this article you will learn how to set up master/slave, install plugins, configure projects and configure variables/properties.
Set up master and slave machines
Jenkins has a built-in SSH client implementation it uses to communicate
sshd and to a slave agent. There are several ways
to communicate between the master and slaves:
- For UNIX slaves, via SSH. You only need SSH and JRE on slaves.
- For Windows, with the Distributed Component Object Model (DCOM)
- Use a separate socket connection when master cannot see slaves, via Java Web Start.
Install master on Linux
To install master on Linux machine type the commands contained in Listing 1.
Listing 1. Install master on Linux
sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key sudo yum install jenkins
Install master on Windows
Type the command in Listing 2 to install master on a Windows machine.
Listing 2. Install master on Windows
java -jar jenkins.war
After you run the command, access http://<hostname>:8080/ then click Manage Jenkins > Install as Windows Service > Install
Set up slaves
Access http://<JenkinsMasterHost>:8080/ > Manage Jenkins > Manage Node > New Node and configure the slave information according to the slave host. Jenkins master helps install Jenkins onto the slave machines.
For the Windows slave, there is an additional command:
Listing 3. Additional command, Windows slave
sc.exe create "<serviceKey>" start= auto binPath= "<path to jenkins-slave.exe>" DisplayName= "<service display name>"
The name of the registry key that defines the service is <serviceKey>. <service display name> is the label that identifies the service in the service manager interface.
Plugins are another important feature in Jenkins. Currently, Jenkins supports more than 1000 plugins. You can divide these plugins into different categories (plugins for source management, for build reports, for build tools, etc.). With plugins, you can monitor, deploy, or configure different jobs in Jenkins. To manage the plugins, go to https://JenkinsMasterHost:8080 then select Manage Jenkins > Manage Plugins. There are four tabs:
- Updates: Installed plugins that have updates
- Available: Plugins available to install
- Installed: Installed plugins
- Advanced: Manage the plugins
Install the plugins via the Internet
When Jenkins master can access the Internet, installing plugins is easy. On the Available tab, choose the plugins to install. You can remove the plugins from the Installed tab. Click the uninstall button to remove the plugins.
Install the plugins manually
You can install the plugins manually if Jenkins master can't access the Internet. Find the plugin that you want to install, then save the downloaded *.hpi/*.jpi file into the $JENKINS_HOME/plugins directory. Restart Jenkins to enable the plugins.
Jenkins supports four types of projects: free-style, maven, multi-configuration and external job. The free-style project is the central feature of Jenkins. It can be combined in SCM with any build system. You can also use it for something other than a software build.
Go to https://JenkinsMasterHost:8080, select New Item, specify the Item name and Type to create the project.
On the project configuration page, the Item name is also called Project name. There are four item types. You can also choose the option copy existing item, as shown in Figure 1. Click OK to open the project configuration page.
Figure 1. Create new item
The information you'll need for the configuration page is:
Project name: If the project name is updated, the item name is also updated.
Description: Job description
Strategy: Log strategy, how many logs to keep
Parameterized: Defines the variables for the project. There are different types of variables (file parameter, text parameter, string parameter, etc.)
Where: Restricts where the project can run
Advance configuration: Further specifications to control how to build the project
The plugins you choose to install affect which categories and functions you'll have in your project. Some categories and functions are:
- Source code management: The tool that manages the source code.
- Build triggers: The method to trigger a build.
- Build: The build is the most important part of a project. You specify the exact steps to run a project. The common steps are DOS commands for Windows, or Shell commands for Linux.
- Post-build actions: The actions after the build. The common actions after the build are: send e-mails, trigger other builds, or publish the results report.
After you complete the configuration, click Save. You'll find the saved project listed in https://JenkinsMasterHost:8080Jenkins > All.
With Jenkins you can trigger to build projects manually or automatically. There are different mechanisms to trigger a build. If you choose to trigger the build automatically, you can define the option in Build Triggers when you configure the project. The options available are:
- Build project after other projects are built: After you configure a project, you can define whether to build other projects after this project. Choose this option if your project is dependent on other projects.
- Trigger builds remotely (e.g., from scripts): The project build is triggered from other system or hosts. For example, you can trigger the project build via email or submit a build request from scripts.
- Build periodically: Create a schedule to build the project periodically, as defined in your configuration.
- Poll SCM: This option builds the project by source changes. With this option, you specify how often Jenkins polls the revision control system. If there are source code changes, the project builds.
Figure 2. Options in Build Triggers
The exact behavior delegation depends on the configuration of each project. If the project is configured as Restrict where this project can run, the project will run on the specified machine only. Other projects may choose to roam freely between slaves, it all depends on the configuration.
Currently, Jenkins employs these strategies to distribute the projects:
- If a project is configured as Restrict where this project can run, it is only run there.
- Jenkins tries to build a project on the same computer that it was previously built.
- Jenkins tries to move long builds to slaves.
Set up variables/properties
You'll set up environment variables (defining the property name and value) or tool locations in the global properties: http://<JenkinsMasterHost>:8080 then select Manage Jenkins > configuration. You can use these properties in all Jenkins projects.
You can refer to environment variables in projects. Select the Environment variables box, and define the name and the value for the variables, as shown in Figure 3.
Figure 3. Define environment variables
Select the Tool Locations check box. Select the tool name from the drop-down and define the home directory for the tool. The tool can be referred to in projects.
Figure 4. Define tool locations
Project local properties
Project local properties are only available within a project. When you configure a project, select the option This build is parameterized as shown in Figure 5. Selecting this option enables the function to add parameters as name/value pairs. You can use these parameters as the project local properties.
Figure 5. Set local properties
Practice: Continuous delivery framework structure and process
Continuous delivery aims to ensure software can be developed, tested, deployed and finally delivered to production in an efficient and high quality approach. With continuous delivery every change to any part of the software system (no matter if it's from the infrastructure, application or customization data level) is continuously applied to the production environment through a specific delivery pipeline.
Continuous delivery requires quick and automatic deployment of change sets. There are several steps to complete a deployment, or delivery. The standard process is:
- The developer delivers changes
- Source control tools make builds
- Run automated tests
- Install the build
The continuous delivery process is shown in Figure 6. At the scheduled time, the first project in Jenkins starts.
- Project 1
- The first task in this project, is to download the source code from a source management tool such as IBM® Rational Team Concert™. If the project fails, failed email notifications are sent and all other projects stop. If the project is successful the next project is triggered.
- Project 2
- IBM® Security AppScan® reviews the source code downloaded from Project 1 for security issues.
- Project 3
- The build starts after the AppScan project completes.
- Project 4
- After a successful build, the next project is to install that build in the Build Verification Test (BVT) environment.
- Project 5
- Run BVT test cases on the BVT environment. If the BVT passes, Jenkins starts to install the build on both development and test environments (Projects 6 and 7).
- Project 6
- Install builds on development environments and sends email notifications. The project install builds on the development environments prepares the environments so the developers can do their integration or development work.
- Project 7
- Install builds on test environments. After the install build on the test environments, Jenkins triggers Project 8 to run Functional Verification Test (FVT).
- Project 8
- FVT is an automation tests list that includes many tests. After the FVT passes, the build is installed on the production environments (Project 8).
- Project 9
- The production environments can be installed on your customer's local servers either on the Cloud, or IBM® Softlayer®. Access is available to both internal and external users.
Figure 6 shows the continuous delivery framework process. After a project succeeds, the next project is triggered. If a project fails, the process ends and emails are sent to subscribers.
Figure 6. Process for the continuous delivery framework
Deployment topology for the continuous delivery framework
The left side of Figure 7 shows a traditional development deployment. A developer commits the change sets to the source control server for example Rational Team Concert, then the build server makes the builds.
The right side of Figure 7 illustrates the continuous delivery process. After Jenkins is added, there is a Jenkins master. The Rational Team Concert buildtoolkit is installed on that server. Jenkins master is installed in a Rational Team Concert plugin. It uses the buildtoolkit to download the source code from Rational Team Concert and also triggers the buildtoolkit to make the build. The AppScan and BVT projects also run on the Jenkins master. The development environments, test environments and production environments all serve as Jenkins slaves. They are controlled by the Jenkins master and they run the installation projects. The test environments run the functional verification test project.
It's easier to keep track of tasks if you tie the projects to the computers because the different machines have different roles.
Figure 7. Topology for the continuous delivery framework
You now know how to set up a continuous delivery framework with Jenkins. This framework helps to deploy work automatically which saves the developers and testers valuable time. The framework also helps you discover any issues or defects early in the process.