Are you a developer of WebSphere Liberty Java Enterprise (JEE) applications wishing to exploit the continuous delivery tools of Bluemix DevOps? In this blog post, we’ll briefly review the basics of Continuous Delivery (CD) Pipeline of JEE Bluemix applications and then share with you a solution for handling any JEE project, particularly those having dependencies to other projects or using common reusable libraries.
CD Pipeline Overview
Users of Bluemix DevOps Services can create a project to build and deploy applications. Such DevOps project is associated with one Git source code repository and (optionally) one continuous delivery (CD) pipeline. If you’re new to Bluemix DevOps Services, see the list of tutorials. If you only need more background for the topic of this post, see Set up a pipeline and deploy an app.
For simple projects the Build and Deploy stages are almost out of the box:
a clone of the Git repository is created within a working directory (on each commit to the repository, or started by the user from the DevOps project UI)
your build script is executed (Ant, Maven etc.)
‘cd’ into a folder within the working directory created in the previous step.
‘cf push’ that is a CloudFoundry (CF) command that pushes a file or folder onto an application.
You may have multiple Deploy stages in your pipeline. Each Deploy stage may be linked to a previous Deploy stage, but often they will define their input to be the output directory of the Build stage as depicted below:
The Deploy stage of a typical pipeline deploys the app into a test space and it is followed by a Test stage that determines whether the previously deployed app is functioning correctly. This is then followed by another stage that pushes the same application into production.
General CD Pipeline for JEE Projects
The above infrastructure is well aligned with the micro-services philosophy requiring relatively small DevOps projects, each one attached to its own Git repository. However, there are at least two situations in which it would be better for a build to get files from multiple repositories:
In order to build a JEE component one needs to access JAR files defining the JEE API’s. If using Liberty, all files under the wlp/dev/api folder of WebSphere Liberty generally need to be available.
A micro-service may assemble a specific function with one or more reusable components (utility or EJB modules), the latter being held inside distinct repositories.
If you are used to developing and testing JEE applications locally on your desktop using your favorite IDE, like JEE Eclipse, you might have paid little attention to the build process. Indeed, the IDE normally performs the build behind the scenes on every update of the source code, using information configured in multiple related projects contained in the user workspace. Unfortunately, this approach is not easily replicated outside of that particular IDE, therefore Bluemix DevOps supports more generic build tools like Ant, Gradle, Maven and others. The build processing starts by executing the shell script whose content you specify in the stage configuration panel.
Let’s see how it is possible to customize the stages for our purposes.
Build stage configuration
If your build needs to assemble different components, you want to avoid entering complex script manually (in the ‘Build Shell Command’ field). Instead, we suggest that you code and maintain the build script as a file stored at the root of the Git repository, while the Build stage configuration script only contains a single instruction executing the copy of your script file extracted from the repository. When necessary, your script will delegate work to other scripts.
With this approach, the Build stage of any such pipeline is identical to the one shown in the following figure:
Standardizing the name (devops_build.sh is our suggested example) allows you to create pipelines for new projects using copy and paste.
Structure of the proposed build script
The build script we propose must meet a simple requirement: Successfully running the script in the Bluemix DevOps environment must produce one or more deployable artifacts at a specific place within the working directory automatically created in the Build stage. With this goal in mind, the script implements the following steps:
Extract the following additional required resources from distinct GIT repositories containing:
(a) Liberty server runtime, which is needed at compile time, (b) Liberty server instance, which is needed if you are pushing a server instance at deploy time (eg. to deploy a customized server.xml), and (c) any reusable software component, such as, clients of Bluemix services.
Build the software components extracted on the previous step, by running the ANT build.xml file contained in each of them
Move the deployable artifacts (EAR or WAR files) created in the previous step to a copy of the Liberty server instance extracted in the first step
Copy into the Liberty server instance the configuration files tailored for the specific needs of the JEE application being deployed, typically server.xml, server.env and jvm.options.
The devops_build.sh script file illustrated in the following section relates to a J2EE project where the application EAR depends on two utility modules (com.bluemix.utils.jar, com.sample.utils.jar) and one EJB module (com.sample.utils.ejb.jar) from additional repositories.
Because the devops_build.sh is purely sequential logic, we can see how straightforward it is to clone the script and adapt them for use in different projects.
Accessing multiple Git repositories
As you can see, there are three Git repositories, beyond the one associated with the pipeline itself, whose content is extracted at the beginning of the Build stage. Repository Rep1 contains a folder we named wlpInstance.template with both Liberty libraries (for compiling stages) and an almost empty instance of the Liberty server (needed when exploiting the most flexible option for deploying apps to Liberty). Repositories Rep2 and Rep3 contain source code and they are examples of common components (bluemix.utils and sample.utils are Eclipse projects in their respective repositories) reusable across different applications.
To adapt the script to your needs you have to fill the user and token variables for proper Git access (The pipeline handles authentication only for the main repo).
Note: The git clone in those commands is a placeholder for a faster alternative like git clone -b master --single-branch --depth 1 that extracts only the last level of code.
Structure of the ANT script
Everyone using Eclipse or NetBeans can have JEE artifacts automatically built by the IDE, however one needs more generic tools for building in the automated pipeline environment. Here we attach below a sample ANT build.xml file (the one referenced in the script above) that builds the JEE application with both local code and JAR artifacts created from previously compiled dependencies. Pay attention that Web applications would require a slightly different build file.
The deploy stage configuration
After a successful build execution, build artifacts are archived in the output directory for the deploy steps. Moreover the Deploy stage can be configured with a custom script. In the case of our sample project, just described so far, one can simply issue the push command below specifying the server directory (wlpInstance in our sample) already containing our application (EAR file) and optionally a custom Liberty server configuration (server.xml and/or server.env):
thanks for your interest. Unfortunately we have not such full sample project ready, I’ll try to manage to cut down some real projects where we applied the approach and post on a public git. This was already done with the two distinctive sample build components already attached but I understand also the boilerplate code around could help. I’ll post here in case.
Liberty and Eclipse make a great local development environment for developing and deploying Java EE applications to Bluemix, especially to the Liberty for Java instant runtime. To help make developing for Bluemix even easier, use the IBM Eclipse Tools for Bluemix. In this post, we’ll explore the IBM Eclipse Tools for Bluemix and how to install them.