Contents


Use management APIs and Jenkins as a continuous integration engine for IBM App Connect Professional deployment automation

Comments

In this tutorial, you will learn about deployment automation on IBM App Connect (formerly known as WebSphere® Cast Iron®) with IBM App Connect Management APIs, the SoapUI client, and Jenkins.

Benefits

Deployment automation:

  • Fully automates the deployment of IBM App Connect orchestrations.
  • Saves effort and time deploying through the Cast Iron admin console.
  • Automates deployment leads to end-to-end tractability.
  • Completes tractability of deployment activities and increases deployment quality, decreasing the downtime of the applications and environments.

Prerequisites

To follow along, you need:

  • IBM App Connect 7.5.0 or higher
  • GitLab or GitHub, so you can check in the PAR and deployment property files (a GitLab account was used here)
  • SoapUI 5.2.1 or higher
  • The latest version of Jenkins
  • Basic knowledge of DevOps

IBM App Connect Professional (formerly known as Cast Iron) is a platform that integrates cloud-based SaaS applications with on-premises applications. It is a drag-and-drop development tool used to build complex integration flows.

IBM App Connect has two implementation models:

  • Appliance model— The appliance can be either physical hardware or a virtual machine.
  • SaaS model (IBM App Connect Professional SaaS)— Each of the three form factors (physical, virtual, SaaS) allows for cloud-to-cloud, cloud to on-premises, and on-premises to on-premises integration. Each form factor allows for real-time, near real-time, and batch integrations.

To access the IBM App Connect runtime, use any of these interfaces:

  • Management API (web services)
  • Command-line interface (CLI)
  • WMC

Appliance APIs

The appliance management API is a web service layer that allows administration of the IBM App Connect runtime by using web service calls in the form of SOAP requests to perform tasks, including:

  • Publish
  • Start
  • Stop
  • Undeploy, delete
  • Export configurations
  • Project

The response is in the form of a SOAP response.

The IBM App Connect integration appliance provides different APIs to perform various functions related to IBM App Connect administration. These APIs are accessible through:

  • Security.wsdl
  • Staging.wsdl
  • Deployment.wsdl
  • System.wsdl
  • Orcmon.wsdl
  • Lognotif.wsdl

The deployment WSDL contains information pertaining to publishing a PAR file, starting a PAR file, stopping an orchestration, undeploying an orchestration, deleting an orchestration, exporting projects, exporting configurations, and so on. Download the https://<appliance-management-port>/ws/deployment?wsdl deployment WSDL in the SoapUI.

The security WSDL contains login and logout operations to establish a management API session. Download the https://<appliance-management-port>/ws/security?wsdl security WSDL in the SoapUI. The details are described in the next sections.

Configuration of IBM App Connect deployment automation

The automation configuration of IBM App Connect deployment consists of two main processes:

  • Configuring a test suite using SOAPUI to call appliance APIs (login, publish, logout, and so on) in sequence.
  • Automating the above invocation of the test suite created by Jenkins.

Configuration of Test Suite to invoke appliance API

The basic steps to publish a PAR file and start the orchestration are shown in Figure 1. You will log in to IBM App Connect, upload the PAR file and deploy it, start the orchestration of this PAR file, then log out of IBM App Connect. The detailed steps to run this are described below.

Note: This flow diagram remains the same whether the deployment is through IBM App Connect WMC or App Connect Management APIs.

Figure 1. Flow diagram of deployment steps
Deployment steps flow
Deployment steps flow

Step 1. Create a SOAP project on SoapUI

Open the SoapUI and create an empty project by clicking File > Create Empty Project. By default, the project is named Project1. You can rename the project whatever name you like. For this tutorial, the project is renamed CastIron_Deployment.

Another way to create a new SOAP project is by clicking File > New SOAP Project. Type the name CastIron_Deployment in the dialog box that opens.

Step 2. Add WSDL for security and deployment

Right-click the project CastIron_Deployment and add security and deployment WSDLs (shown in figures 2a, 2b, and 2c). Figure 2a shows the first step, which adds WSDL to a project. Right-click the CastIron_Deployment project and click Add WSDL.

Figure 2a. Adding WSDL to a project
CastIron_Deployment project right-clicked and Add WSDL selected
CastIron_Deployment project right-clicked and Add WSDL selected

The Add WSDL dialog box opens. To add security, type (or browse to) https://appliance-management-port/ws/security?wsdl in the WSDL Location text box, as shown in Figure 2b.

Figure 2b. Add security WSDL
Add WSDL dialog box
Add WSDL dialog box

You'll also want to add deployment WSDL. Right-click the CastIron_Deployment project and click Add WSDL. The Add WSDL dialog box opens. To add deployment, type (or browse to) https://appliance-management-port/ws/deployment?wsdl in the WSDL Location text box, as shown in Figure 2c.

Figure 2c. Add deployment WSDL
Add WSDL dialog box
Add WSDL dialog box

Step 3. Create a test case

Right-click the project CastIron_Deployment and select New TestSuite, as shown in Figure 3a.

Figure 3a. Add a new TestSuite
NewTestSuite selected
NewTestSuite selected

As before, right-click on TestSuite1 and select New TestCase, as shown in Figure 3b.

Figure 3b. Add a test case
NewTestCase selected
NewTestCase selected

By default, the new test case is named TestCase1. Rename this to PublishandStart by right-clicking TestCase1 and selecting Rename, as shown in Figures 3c.

Figure 3c. Rename a test case
Right-click and rename to PublishandStart
Right-click and rename to PublishandStart
Right-click and rename to PublishandStart
Right-click and rename to PublishandStart

Step 4. Upload the PAR and property files to GitLab

Next, you need to upload the PAR file and the deployment property file to the GitLab account. A property file has the extension .properties. Property files are mainly used in Java™ technologies to store the configurable parameters of an application. Each line in a .properties file usually stores a single property in key-value pair format. Figure 4 shows the deployment property file that is named publishPar.properties with the properties:

  • projectName
  • version
  • config
  • FileName

This file should be created and uploaded to GitLab.

Figure 4. Deployment property file (publishPar.properties)
projectName=#Generic Properties TestDatabase, version=2.0,                 config=Default, FileName=TestDatabase.par, #Generic Properties End
projectName=#Generic Properties TestDatabase, version=2.0, config=Default, FileName=TestDatabase.par, #Generic Properties End

Step 5. Define the test steps

The test steps subdivided to:

  • Step 5a: Login. This step logs in a user to the IBM App Connect appliance.
  • Step 5b: Attach deployable PAR file on SoapUI. During this step, the deployable PAR file is made accessible to the test case.
  • Step 5c: Transfer the SessionId generated. In IBM App Connect, a user is authenticated before performing any administration activities. If you want to perform any other activities (such as publish or logout), after the login, the user must be authorized. Therefore, a SessionId must be transferred to invoke any operation (or test case) for a specific user session.
  • Step 5d: Publish the PAR file. This step deploys the PAR file in IBM App Connect.
  • Step 5e: Start the orchestration. After the PAR file is deployed in Step 5d, the orchestration in IBM App Connect might start — or might not — depending on the requirement. This step starts the orchestration in IBM App Connect, if required.
  • Step 5f: Log out of current session. This step logs out a user.

Step 5a. Login

In the SoapUI navigator, right-click Test Steps. In the pop-up menu, click Add Step then SOAP Request, as shown in Figure 5.

Figure 5. Add a SOAP request in SoapUI
Test Steps > Add Step > SOAP Request
Test Steps > Add Step > SOAP Request

The New TestRequest window opens. Select the SecurityManagerPortBinding > login option from the list, then click OK, as shown in Figure 6.

Figure 6. Select Login Appliance API Request from SecurityManagerPortBinding -> login
New TestRequest: Select operation to invoke for request: SecurityManagerPortBinding -> login
New TestRequest: Select operation to invoke for request: SecurityManagerPortBinding -> login

Add the username between the <sec:username> </sec:username> tags. Add the password between the <sec:password> </sec:password> tags, as shown in Figure 7. Click Save to save the project.

Figure 7. Login SOAPRequest
LoginSOAPRequest username and password
LoginSOAPRequest username and password

If successful, this login request generates a SessionId for user verification. This SessionId is passed to the rest of the steps.

Step 5b. Attach deployable PAR file on SOAPUI

Right-click Test Steps, then click Add Step. Select Groovy Script, as shown in Figure 8. This script uploads the PAR file in the SoapUI from a specified location. The script uses the deployment property file you uploaded to GitLab, shown in Figure 4.

Figure 8. Adding Groovy Script
Test Steps > Add Step > Groovy Script
Test Steps > Add Step > Groovy Script

Copy and paste the script below into the Groovy script window so the SoapUI can access and upload the PAR file to publish to IBM App Connect:

import org.custommonkey.xmlunit.* 
import java.util.Random   
import java.security.MessageDigest 
import java.nio.file.* 


    def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context) 
    def projectPath = groovyUtils.projectPath 
    log.info projectPath 
    Properties properties = new Properties() 
    File propertiesFile = new File('C:\\Jenkins\\workspace\\CastIronDeployment\\publishPar.properties') 
    propertiesFile.withInputStream {properties.load(it)} 
     
    def project = testRunner.testCase.testSuite.project 
        log.info "Project: " + project.name 
        def myTestSuite = testRunner.testCase.testSuite; 
        log.info "TestSuite: " + myTestSuite.name 

    def testCase = testRunner.testCase 
    log.info "TestCase: " + testCase.name 

    def testStepUploadDataAfterCheck = testCase.getTestStepByName("PublishSOAPRequest") 
    def request= testStepUploadDataAfterCheck.testRequest 
    log.info "TestStep: " + testStepUploadDataAfterCheck.name 


    // clear existing attachments 
    for( a in request.attachments ) { 
        request.removeAttachment( a ) 
    } 

     
    def propFileNamePath='FileNamePath' 
    def propFileName='FileName' 
    //FileNamePath 
    //def fileNamePath = testCase.getTestStepAt(testRunner.testCase.getTestStepIndexByName("FileNameProperties")).getProperty("FileNamePath") 
    def fileNamePath = properties."$propFileNamePath" 
   	//log.info fileNamePath 
     //log.info properties."$propFileNamePath" 
    //FileName 
    //def fileName = testCase.getTestStepAt(testRunner.testCase.getTestStepIndexByName("FileNameProperties")).getProperty("FileName") 
    def fileName = properties."$propFileName" 
    //log.info fileName 

    // get file to attach 
   // log.info "file to attach: " + fileNamePath.getValue() 
   log.info "file to attach: " + fileNamePath 
    //def file = new File(fileNamePath.getValue() ) 
    def file = new File(fileNamePath) 
    if ( file == null) { 
        log.error "bad filename" 
    }    
    else 
    { 
        // attach and set properties 
        def attachment = request.attachFile( file, true ) 
        attachment.contentType = "application/octet-stream" 
        //attachment.setPart(fileName.getValue()) 
        attachment.setPart(fileName) 

        def holder2 = groovyUtils.getXmlHolder( "PublishSOAPRequest#Request" ) // Get Request body 
        //holder2.setNodeValue( "//dep:publishProject/content","cid:"+fileName.getValue()); //Set "link" to attachment in request body 
        holder2.setNodeValue( "//dep:publishProject/content","cid:"+fileName); //Set "link" to attachment in request body 
        holder2.updateProperty() //and update 
        //log.info fileName.getValue() 
        log.info fileName 
        log.info "file attached succesfully" 
    }

Step 5c. Transfer the SessionId generated

Now you need to transfer the SessionId obtained in step 5a, then passed in steps 5d, 5e, and 5f.

Right-click Test Steps, select Add Step, then select Property Transfer, as shown in Figure 9. This transfers the SessionId of the login response to all other test steps.

An XPATH query transfers the SessionId through the requests from login response to publish (Step 4d), start (Step 4e), and logout (Step 4f).

The figure below shows how to pass the SessionId from the login step to the logout step (shown in the Figure 11a). Similarly, how SessionId is to be transferred to publish step shown in Figure 11b and start the orchestration step, shown in Figure 11c.

Figure 9. Add step: property transfer
Test Steps > Add Step > Property Transfer
Test Steps > Add Step > Property Transfer

Click the + icon (highlighted in red in Figure 10) to add a property transfer. This opens a dialog box. Type the name transferSessionId in the Specify name for value transfer text box. You will repeat this step three times, as shown in figures 11 a, b, and c.

Figure 10. Add transfers in the property transfer step
Property Transfer > Add Transfer: Name for value transfer                 transferSessionId
Property Transfer > Add Transfer: Name for value transfer transferSessionId
Figure 11a. Property transfer XPATH query for SessionID from login step to logout step
Property Transfer Target: LogoutSOAPRequest, Property: Request, Path                 Language: XPath
Property Transfer Target: LogoutSOAPRequest, Property: Request, Path Language: XPath
Figure 11b. Property transfer XPATH query for SessionID from login step to PublishSOAPRequest step
Property Transfer Target: PublishSOAPRequest, Property: Request, Path                 Language: XPath
Property Transfer Target: PublishSOAPRequest, Property: Request, Path Language: XPath
Figure 11c. Property transfer XPATH query for SessionID from login step to StartRequest step
Property Transfer Target: StartRequest, Property: Request, Path                 Language: XPath
Property Transfer Target: StartRequest, Property: Request, Path Language: XPath

Step 5d. Publish the PAR file

To publish the PAR file, you first need to add a test step as a SOAP request, as shown in Figure 5. After the New TestRequest window opens, select DeploymentManagerPortBinding > publishProject from the list, as shown in Figure 12).

Figure 12. Publish request
NewTestRequest DeploymentManagerPortBinding ->publishProject
NewTestRequest DeploymentManagerPortBinding ->publishProject

The PublishSOAPRequest is shown in Figure 13.

Figure 13. PublishSOAPRequest
PublishSOAPRequest
PublishSOAPRequest

Step 5e. Start the orchestration after deployment completes (optional)

Depending on the requirement, after the publish step, you might need to start the orchestration. If the orchestration does not start, the IBM App Connect WMC shows that the project status is undeployed. If the orchestration successfully starts, orchestration shows a status of running.

If it is required that orchestration starts, you need to enable the start appliance call. Otherwise, you must disable the start API call. To accomplish this, right-click Test Steps, click Add Step, select Groovy Script, as shown in Figure 8, then add the Groovy Script below to enable or disable the start call:

Properties properties = new Properties() 
File propertiesFile = new File('C:\\Jenkins\\workspace\\CastIronDeployment\\publishPar.properties') 
propertiesFile.withInputStream {properties.load(it)} 
def enableStart='enableStart' 

if (properties."$enableStart"=="true"){ 
	def testStep = testRunner.testCase.getTestStepByName( "StartRequest" ) 
	log.info testStep.disabled 
	if(testStep.disabled){ 
		testStep.disabled = false 
		} 
	log.info testRunner.testCase.getTestStepByName( "StartRequest" ).disabled 
} 
 else if (properties."$enableStart"=="false") { 
	log.info "Orchestration will not be started" 
	def testStep = testRunner.testCase.getTestStepByName( "StartRequest" ) 
	log.info testStep.disabled 
	if(!testStep.disabled){ 
		testStep.disabled = true 
		} 
	log.info testRunner.testCase.getTestStepByName( "StartRequest" ).disabled 
	} 
else 
	log.info "Incorrect value for enableStart in properties file."

Right-click the Test Steps and add the new test step as a SOAP request. Select the DeploymentManagerPortBinding > start request from the list, as shown in Figure 14.

Figure 14. Start request
NewTestRequest: DeploymentManagerPortBinding ->start
NewTestRequest: DeploymentManagerPortBinding ->start

Figure 15 shows the management API call to the start orchestration.

Figure 15. SOAP request to start the orchestration
StartRequest ManagementAPI call
StartRequest ManagementAPI call

Step 5f. Log out from the current session

Right-click the Test Steps and add the new test step as a SOAP request, as shown in Figure 5. Select the SecurityManagerPortBinding > logout request, as shown in Figure 16.

Figure 16. Logout request
NewTestRequest SecurityManagerPortBinding ->logout
NewTestRequest SecurityManagerPortBinding ->logout

Figure 17 shows the LogoutSOAPRequest dialog.

Figure 17. SOAP Logout request
LogoutSOAPRequest
LogoutSOAPRequest

Figure 18 shows the final test suite configuration that includes the seven test steps:

  • LoginSOAPRequest
  • UploadDataAfterCheck
  • Property Transfer
  • PublishSOAPRequest
  • SelectTestStep
  • StartRequest
  • LogoutSoapRequest
Figure 18. SoapUI project navigator with test steps
Projects: TestCase1: Test Steps: StartRequest highlighted
Projects: TestCase1: Test Steps: StartRequest highlighted

Step 6. Export the SoapUI project with the test suite

Right-click the SoapUI project and click Save Project. When prompted for a file name, name the test suite CastIron-Deployment-soapui-project.xml, as shown in Figure 19a.

Figure 19a. Save a SoapUI project
Save Project selected
Save Project selected

Figure 19b shows the CastIron-Deployment-soapui-project.xml file uploaded to GitLab.

Figure 19b. CastIron-Deployment-soapui-project.xml on GitLab
CastIron-Deployment-soapui-project.xml file screenshot from GitLab
CastIron-Deployment-soapui-project.xml file screenshot from GitLab

Automation of invocation of test suite through Jenkins CI

Now it is time to use Jenkins as a continuous integration (CI) engine to orchestrate the deployment automation.

Step 1. Configure SOAP UI shell script

The first step is to change the SoapUI default shell script ({SOAPUI_HOME}/bin/ testrunner.sh) to run with a property file. In this case, change it to run the publishPar.properties property file created by a Jenkins job in the subsequent steps.

Figure 20. Change testrunner.sh to include a property file
changing testrunner.sh to include a properties file
changing testrunner.sh to include a properties file

Step 2. Configure Jenkins job to run the SoapUI project

A Jenkins job is created to run the SoapUI project.

Step 2a. Input deployment properties

Set the builds required parameters to:

  • targetenv: The target environment of the build.
  • propertyFileName: Property file name that is uploaded in the Git repository.
  • Filename: The PAR filename which must be published in IBM App Connect.
  • enableStart: True=Start orchestration after publishing the PAR file. False= Do not start the orchestration to after deployment of the PAR file.
Figure 21. Required parameters
Jenkins build pararmeters
Jenkins build pararmeters

Step 2b. Download PAR file and property file from GitLab

In Jenkins jobs, click the job name (for this tutorial, the job name is CastIronDeployment_dev).

Figure 22a. Click the Jenkins job name
CastIronJobs in Jenkins with CastIronDeployment_dev selected
CastIronJobs in Jenkins with CastIronDeployment_dev selected

Click Configure.

Figure 22b. Jenkins project CastIronDeployment_dev job
Jenkins screen with configure selected
Jenkins screen with configure selected

The configure window opens. Select the Source Code Management tab, then select the Git radio button. In the Repository URL field, type the link to the repository in Git.

Figure 22c. Set SCM to GitLab
Set Source Code Management to GitLab URL and credentials
Set Source Code Management to GitLab URL and credentials

Step 2c. Create a property file that combines deployment properties with the base property file during runtime

Click the Build tab. To create a new property, run the following shell script (lines 1-10) in Figure 23. The script (Jenkins Shell Script.txt) is available in the downloads section of this tutorial.

Figure 23. Create a property file with a Jenkins script
Jenkins script Jenkins Shell Script.txt
Jenkins script Jenkins Shell Script.txt

Figure 24 shows the combined property file at runtime. Notice both the properties by the development and deployment teams are listed.

Figure 24. Combined property file
Combined property file with properties by development team and                 properties by deployment team
Combined property file with properties by development team and properties by deployment team

Step 2d. Run the SoapUI test suite using shell script

Figure 25 shows the Jenkins Shell Script.txt shell script (available in the downloads section). You'll add this script in the section under Build (lines 13-15).

Figure 25. Run shell script
Execute shell script
Execute shell script

Step 2e. Backup deployment artifacts

On the Post-build Actions tab, you'll see an archive the artifacts section. The artifact PAR file, property file, and log files in the file system will be archived. You can access the archives at {JENKINS_HOME}/jobs/{JENKINS_JOB_NAME}/builds/{JENKINS_JOB_NO.}/archive.

Figure 26. Store post-build artifacts
Post-build actions with archive the artifacts and files to archive
Post-build actions with archive the artifacts and files to archive

Conclusion

This tutorial showed how IBM App Connect management APIs provide a flexible way to administer the appliance and orchestrations. IBM App Connect is easily integrated with DevOps tools to facilitate deployment automation and configure the delivery pipeline.

Files used in this tutorial


Downloadable resources


Related topics


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=DevOps, Open source
ArticleID=1056978
ArticleTitle=Use management APIs and Jenkins as a continuous integration engine for IBM App Connect Professional deployment automation
publish-date=02052018