Managing remote repositories on a dynamic cloud network using IBM UrbanCode Deploy

Learn how to build a remote package repository for IBM UrbanCode Deploy to use in the context of a continuous delivery process. This will optimize and drastically reduce the delivery time of the packages. The solution is based on a real case study for the DevOpsInACloud project. Because the DevOpsInACloud environments are located on different networks, the download of a new version of the product on each agent might be time consuming. Learn how to create and configure a remote repository for each network and configure the UrbanCode Deploy processes to download the build artifacts from these repositories.

Share:

Ilaria Gorga (ilaria.gorga@it.ibm.com), Software Engineer , IBM

Author1 photoIlaria Gorga has worked as a software engineer in the IBM Tivoli Rome Lab since 2008. She has been on the Tivoli Workload Scheduler quality assurance team since 2010 and focuses on test automation optimization. Ilaria has written several other articles and presented at conferences, such as Eclipse-IT.



Fabio Barillari (fabio.barillari@it.ibm.com), Senior Software Engineer , IBM

Photo of BarillariFabio Barillari is a senior software developer and designer. He works as Quality Assurance Architect at IBM. He defined DevOps processes for IBM Workload Automation.



20 May 2014

Also available in Russian

Introduction

To optimize and drastically reduce the time required to deliver packages in a continuous delivery process you can build a remote package repository that IBM® UrbanCode® Deploy can use. The solution is based on a real case study, the DevOpsInACloud scenario. The DevOpsInACloud environments are located on different networks; therefore, the download of a new version of the product on each agent might be time consuming. This article describes how to create and configure a remote repository for each network and configure the UrbanCode Deploy processes to download the build artifacts from these repositories.

The DevOpsInACloud architecture is shown in Figure 1.
UrbanCode Deploy server, repository on 2 networks

The communication between Network A and Network B is implemented through a secure virtual private network (VPN)connection. This type of connection is slower than a local connection.

The staging and production environments are composed by sub-environments. Each sub-environment contains two nodes: a primary node and a secondary node. The goal of the DevOpsInACloud project is to keep all of the environments up to date.

Each time a new update is necessary, all of the agents need to download the new version of the components to the local environment. For example, to update two components on ten machines, twenty separate downloads are started from UrbanCode Deploy repository to agent on Network B. These concurrent downloads create a slow connection and the process is time consuming.

This article describes how to define a remote repository on Network B and configure the UrbanCode Deploy processes to download the new versions from this repository.


Proposed solution

The proposed solution is based on a specific configuration of UrbanCode Deploy processes and the use of an HTTP server. Install an HTTP server on the machine in Network B that is to be used as the remote repository.

In the architecture shown in Figure 1, the HTTP server is installed on the same machine where the UrbanCode Deploy relay is installed.

Use these steps to configure UrbanCode Deploy processes to use the remote repository:

  1. Define an environment with the remote repository.
  2. Define the processes to download the artifacts to the remote repository.
  3. Define and configure a new version status.
  4. Configure an automatic flow to download the build to the remote repository.
  5. Define processes to download the artifacts from remote repository.

Step 1. Define an environment with the remote repository

Install an agent on the remote repository machine and connect it to the relay.

After the agent is installed, create an environment in your application that includes this agent, as shown in Figure 2.

Figure 2. List of application environments
Two environments listed under DevOpsInACloud app

Click to see larger image

Figure 2. List of application environments

Two environments listed under DevOpsInACloud app

Associate all the components that need to be deployed on Network B to the agent, as shown in Figure 3.

Figure 3. Environment definitions
Application environment details

Click to see larger image

Figure 3. Environment definitions

Application environment details

Step 2. Define the processes to download the artifacts

Define a component process for each component that needs to be deployed on Network B. This process makes it possible to download the specific build artifact to the remote repository.

Figure 4 shows how the step Download Artifacts is configured to copy the new component version to the remote repository.

Figure 4. Download Artifacts step details
Settings for the Download Artifacts step

The destination path of the artifact on remote repository is:

< REMOTE_REPOSITORY_PATH >/< COMPONENT_REP_DIR >/<VERSION_NAME>
  • REMOTE_REPOSITORY_PATH: The base path used to locate files for use by the HTTP server. It is defined as an environment property because it is common to all components.
  • COMPONENT_REP_DIR: A specific subdirectory of the component in the repository. It is defined as component property.
  • VERSION_NAME: A specific subdirectory relative to the specific version. It is a built-in property.

Step 3. Define and configure a new version status

All of the deployment processes that run on the environment located in Network B can be run only if the version to deploy is still present on the remote repository. To satisfy this requirement, you must define a new version status and set it to a value that represents each version that is moved to the remote repository.

Click Settings > Plugins > Status to define a new version status Ready_to_download, as shown in Figure 5.

Figure 5. Version status definition
Description of each status

Click to see larger image

Figure 5. Version status definition

Description of each status

Define another component process for each component, to set the new version status each time the version is downloaded to the remote repository, as shown in Figure 6.

Figure 6. Add status to version step
Edit properties panel to add status

Click to see larger image

Figure 6. Add status to version step

Edit properties panel to add status

The next step is to define the component process. Define an Environment Gate for all of the environments on Network B. Click Application > Configuration > Environment Gates and specify Ready_to_download as the gate for the environment on Network B, as shown in Figure 7.

With the gate in place, a version is applied to an environment only if the version has the status Ready_to_download.

Figure 7. Environment gate in the application definition
Settings for environment gates

Configure an automatic flow to upload the build

Moving the versions from Network A to Network B is a time-consuming activity. A better option is to start an automatic process to move the new version to remote repository.

Define a new application process for each component that needs to be deployed on Network B. This process includes two steps, as shown in Figure 8:

  • Upload the build to the remote repository using the component process defined in Step 2.
  • Promote the version that has the status Ready to download using the component process defined in Step 3.
Figure 8. Application process definition
Workflow: Download and promote Component1

To run the process for each component each time a new version is available, click the Configuration tab, select the check box Run process after creating a new version, and specify the environment and the process is to be run, as shown in Figure 9.

Figure 9. Component configuration
Define basic settings for Component1

Define processes to download the artifacts

The last step is to configure a component process to download the artifact from the remote repository. You can use this component process as the first step of each deployment process on Network B for each component.

This component process must be configured to download the version from the repository only once. If that version is already present on the machine, this step must be ignored.

The process is illustrated in Figure 10.

Figure 10. Component process definition
Workflow Check build. Download build

The process includes three steps:

Verify the build is present on the resource

In the CheckBuildAlreadyDownloaded step, verify that the build is already present on the resource as shown in Figure 11. The COMPONENT1_BUILD_PATH is a property defined at component level and represents the local path on the current resource where the Component1 artifacts are downloaded.

This step has a post-processing script to set the result of the operation in a property that can be read from other steps in the process. The post-processing script is defined in Listing 1.

Listing 1. Post-processing script
properties.put("result", "default");
if (properties.get("exitCode") != 0) {
properties.put(new java.lang.String("Status"), new java.lang.String("Failure"));
}
else {
properties.put("Status", "Success");
scanner.register("Success", function(lineNumber, line) {
var temp=line;
properties.put("result", "true");
});
scanner.register("Fail", function(lineNumber, line) {
var temp=line;
properties.put("result", "false");
});
scanner.scan();
}
Figure 11. Check to verify that the build is already present
CheckBuildAlreadyDownloaded step

Download the build

In the DownloadBuild step, the build is downloaded from the repository to the agent host. The IP address of the remote repository is defined as variable. Using this variable, each environment can have a specific repository and the process can be used for all of the environments. This step has a precondition that evaluates the result of the previous step. If the build is already present, the DownloadBuild step is ignored, as shown in Figure 12.

Figure 12. Step to download the build from the remote repository
Set the precondition in the DownloadBuild step

Delete the partial build if the build fails

In the DeleteVersion step, if the DownloadBuild step fails, the partial version is deleted from the machine.


Conclusion

A continuous delivery solution in a cloud environment requires managing environments located on different networks. In the case of the DevOpsInACloud project, the issue of managing the repositories is solved by implementing an solution based on UrbanCode Deploy and system configurations.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into DevOps on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=DevOps
ArticleID=971565
ArticleTitle=Managing remote repositories on a dynamic cloud network using IBM UrbanCode Deploy
publish-date=05202014