Continuous integration setup for IBM System Planning Tool

Nightly build setup

This article describes the detailed steps to configure your own continuous integration build set up for the IBM® System Planning Tool (SPT) without making use of any other publicly available continuous integration tools. This setup comes with the scheduled code extraction and build, scheduled test suite execution, and scheduled code coverage report generation features. You can use this article as a reference while setting up your own continuous integration environment for any of your projects as it gives step-by-step configuration details. This greatly reduces your research, exploration, and development time of the project setup.

Sandhya Rani (saanjana@in.ibm.com), System Software Engineer, IBM India Software Lab

Photo of SandhyaSandhya is a Java Developer and Team Lead for the product called IBM SystemPlanning Tool (SPT). While working with this product, she had explored and set up the continuous integration environment for this project. This work was done as a value add to the quality aspect of the project. With more than seven years of software development experience, Sandhya holds a Master of Science in Computer Science from Christ University and a Bachelor of Engineering in Computer Science from Visvesvaraya Technological University. Sandhya has published three technical papers on International Journal of Computer Applications (IJCA) with topics related to object-persistence methods and load-balancing techniques.



07 March 2014

Also available in Russian Spanish

Introduction

The IBM System Planning Tool (SPT) is a web application tool that helps users to design system configurations according to their needs. Using this tool, a user can configure either a completely new system or can plan a system based on existing performance data or based on new workloads. The final configured plan of a system is called a system plan or sysplan. The sysplan generated can be used by the user to place an order to purchase IBM servers. Apart from this, system plans generated by SPT can also be used to deploy the plan on the physical system by the Hardware Management Console (HMC).

The following sections describe the steps required to configure the continuous integration setup for a SPT project in a Linux® environment but also applies to the IBM AIX® or IBM i operating systems (see the following note).

Note: Readers can use this article as a reference document to set up their own continuous integration environment for their project. Users are required to change the project files, path, source version controller software, and so on according to their project needs. Also, if the users are required to set up the environment in an operating system other than Linux, then they would need to use the respective operating system commands while configuring the setup.


Pseudo code or algorithm of continuous integration setup

  1. Extract the latest source files or the incremental source files from Configuration Management Version Control (CMVC)
  2. Build the extracted files.
  3. If build is successful:
    1. Run the full test suite/bucket and generate the test suite report.
    2. Run the code coverage script and generate the code coverage report.
    3. Send an email about build success with the build log and coverage reports.
  4. If build failed, retry the build two more times after a specific time interval.
    • If build is successful:
      1. Run the full test suite/bucket and generate the test suite report.
      2. Run the code coverage script and generate the code coverage report.
      3. Send an email about build success with the build log and coverage reports.
    • If build failed:
      1. Find the file which caused the build to fail.
      2. Send an email to the concerned developer about the build failure along with the name of the source file that caused failure.
  5. Repeat these steps at the scheduled time every day as given in the crontab entry.

Scheduled code extraction and build

Twice a day, the build script extracts the latest source files from CMVC and builds the source (at 10 a.m. and 5 p.m. everyday).

If the latest build fails, then it retries two more times after a time interval of 1 minute since the first build.

It sends the build status (success/failure) email to the concerned project team so that they can take the appropriate action accordingly.

In case the build fails even after two retries, then it finds the file which caused the build to fail and sends an alert email to the developer, who checked in that file recently, to take the necessary action.

Download the required software and files

You require the following software and files while performing the continuous integration setup for the IBM SPT project. You can either download these files from the Internet or from the Download section.

  • IBM Software Development Kit (SDK) for Java™
  • CMVC command line client for Linux
  • Apache Ant
  • Apache Tomcat server
  • Code Coverage Jar (emma.jar)

Scheduled code extraction and build configuration steps

  1. Download and copy the v1r2m0d project build folder, and the cmvc_Linux_client_commands.tar, apache-ant-1.8.2-bin.tar and emma.jar files to the setup system.

    Note: Here, v1r2m0d is one of the SPT project's folder that contains the complete project source files.

  2. Change the directory to the folder where the CMVC command-line client tar file is copied.
    [root@localhost ~]# cd CMVC_client/
    [root@localhost CMVC_client]# ls
    cmvc_Linux_client_commands.tar
  3. Extract the files of CMVC command line client tar file.
    [root@localhost CMVC_client]#tar -xvf cmvc_Linux_client_commands.tar
  4. Change the directory to bin/, and when you list the files, you can see all the CMVC command files.
    [root@localhost CMVC_client]# cd bin/
    [root@localhost bin]# ls
    Access    Authority   Cfgrelproc  cmvclog    Config    Defect       Feature  Folder    
    Level        Note     Prereq   Release  Size   Track
    Approval  Become      client.cmc  cmvcntrl   Conflict  Environment  File     Host      
    LevelMember  Notify   Process  Report   Stage  User
    Approver  Cfgcomproc  Cmvchost    Component  Coreq     Family       Fix      Interest  
    Migrate      Package  Project  Shadow   Test   Verify
  5. Download and install Apache Ant.

    After download, extract the Ant folder contents using the following commands.

    gzip -d apache-ant-1.8.2-bin.tar.gz
    tar -xf apache-ant-1.8.2-bin.tar
  6. If Java is not installed, then install it and set the classpath.
    • Download IBM SDK for Java and install it.
      ./ibm-java-i386-sdk-7.0-0.0.bin
    • After this, set the path in /etc/profile.d

      Create a file called java.sh and add the following lines into that file.

      vi /etc/profile.d/java.sh
      
      export JAVA_HOME=/opt/ibm/java-i386-70
      	
      export ANT_HOME=/root/apache-ant-1.8.2
      	
      export CMVC_HOME=/root/CMVC_client
      		
      export CMVC_AUTH_METHOD=PW
      export CMVC_FAMILY=xxxx@xxxx.yyyy.ibm.com@1312
      export CMVC_USER=zzzz
      		
      PATH=$PATH:$JAVA_HOME/bin:${ANT_HOME}/bin:$CMVC_HOME/bin:

    Note: Change the path of the files according to the project setup configuration.

  7. Add the CMVC family name and port number to the services file.

    Syntax

    xxxx@xxxx.yyyy.ibm.com 1312/tcp

    Where, xxxx@xxxx.yyyy.ibm.com is the CMVC family name and 1312 is the corresponding CMVC port.

    Append this above line in the /etc/services file.

    nimhub		      48002/tcp			# Nimbus Hub
    nimhub		      48002/udp			# Nimbus Hub
    nimgtw		      48003/tcp			# Nimbus Gateway
    nimgtw		      48003/udp			# Nimbus Gateway
    com-bardac-dw	      48556/tcp			# com-bardac-dw
    com-bardac-dw	      48556/udp			# com-bardac-dw
    iqobject	      48619/tcp			# iqobject
    iqobject	      48619/udp			# iqobject
    # Local services
    xxxx@xxxx.yyyy.ibm.com 1312/tcp

    Comment the lines if they have the same port (for example, in the following file entry, lines with port 1312, which have the same port as that of CMVC family, are commented).

    rxmon              1311/tcp          # RxMon
    rxmon              1311/udp          # RxMon
    #sti-envision                  1312/tcp                      # STI Envision
    #sti-envision 1312/udp # STI Envision bmc_patroldb 1313/udp # BMC_PATROLDB

Running a scheduled test suite and generating a scheduled code coverage report

After every successful build, the build script runs a full test suite (test bucket/test cases) and generates the test suite report for the SPT project.

Also, after every successful build, it runs the code coverage script and generates the code coverage report in the HTML form so that any project team member can download those coverage reports later and check them.

Scheduled code coverage configuration steps

  1. Download and copy the emma.jar (code coverage jar) file into the <your jre dir>/lib/ext/ directory.

    For example: /root/nightly_builds_docs/ibm-java-i386-60/jre/lib/ext/emma.jar

  2. Provide permissions to run the emma.jar file.
    chmod +x emma.jar

Steps for setting up of the Apache server to download the build log, test bucket execution result log, and coverage reports

  1. Download and copy the apache-tomcat-7.0.27 folder to /root/nightly_builds_docs.
  2. Change the directory to the bin folder of the Apache Tomcat folder.
    cd /root/nightly_builds_docs/apache-tomcat-7.0.27/bin
  3. Start the Apache Tomcat server by running the following command.
    ./startup.sh &

Configuring the project build folder, shell scripts, and cronjob

This section describes the steps required for configuring the project build folder, shell scripts, and cronjob for scheduling the code extraction and running the build.

Steps for configuring the project's build folder and shell scripts

  1. Create the cmvc_extract folder under the v1r2m0d project folder, where the CMVC extracted new source files will be placed.
    mkdir /root/nightly_builds_docs/build/v1r2m0d/cmvc_extract
  2. Create the build_home folder under the v1r2m0d project folder where the new source files without the cmvc path will be copied from the cmvc_extract folder and later the files are built.
    mkdir /root/nightly_builds_docs/build/v1r2m0d/build_home
  3. Copy the following shell scripts into the project build folder (/root/nightly_builds_docs/build/v1r2m0d.hpi/) and provide permissions to run the following scripts.
    • send_failed_build_mail.sh
    • nightlyBuild_retry.sh
    • coverage_script.sh
    chmod 777 send_failed_build_mail.sh
    chmod 777 nightlyBuild_retry.sh
    chmod 777 coverage_script.sh
  4. Copy the main shell script, build_coverage_send_mail.sh to the folder /root/nightly_builds_docs/build/ and provide permissions to run this shell script.
chmod 777  build_coverage_send_mail.sh

Configuring the cronjob for scheduling code extraction and running the build

  1. Set up a scheduler / cronjob to run the build script at a specific time of the day.
    1. Install crontab by using the following command.
      [root@localhost v1r2m0d]# crontab -e
      no crontab for root - using an empty one
      crontab: installing new crontab

      Note: It opens a Vi editor where you need to enter the job with timings and then save and close.

    2. Create a crontab entry to run a specific script at the specific time of the day.

      Note: Provide the crontab entries as per your project needs. Some examples are given below.

      The following crontab entry runs daily at 12.05 a.m.

      [root@localhost v1r2m0d]# crontab -l
      5 0 * * * export JAVA_HOME=/root/nightly_builds_docs/ibm-java-i386-
      60;/root/nightly_builds_docs/build/build_coverage_send_mail.sh &

      The following crontab entry runs twice daily, once at 12 midnight and once at 12 noon.

      00 0,12 * * * export JAVA_HOME=/root/nightly_builds_docs/ibm-java-i386-
      60;/root/nightly_builds_docs/build/build_coverage_send_mail.sh &

      The following crontab entry runs for every hour.

      0 * * * * export JAVA_HOME=/root/nightly_builds_docs/ibm-java-i386-
      60;/root/nightly_builds_docs/build/build_coverage_send_mail.sh &

      The following crontab entry runs daily once at 10 a.m. and once at 5 p.m.

      0 10,17 * * * export JAVA_HOME=/root/nightly_builds_docs/ibm-java-
      i386-60;/root/nightly_builds_docs/build/build_coverage_send_mail.sh &

      The following crontab entry runs daily once at 10 a.m. and once at 5 p.m. and sends the crontab output to a text file.

      0 10,17 * * * export JAVA_HOME=/root/nightly_builds_docs/ibm-java-
      i386-60;/root/nightly_builds_docs/build/ build_coverage_send_mail.sh 
      > /root/nightly_builds_docs/build/crontab_output.txt &
  2. Restart the system.

Resources


Download

DescriptionNameSize
Code sampleconfiguration_scripts.zip5 KB

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into AIX and Unix on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX, IBM i, Linux
ArticleID=964848
ArticleTitle=Continuous integration setup for IBM System Planning Tool
publish-date=03072014