Continuous Delivery Pipeline Versioned Base Image support
Developers know that everything—including the tools they use to build applications, the libraries and packages their applications depend on, and even the environments where the applications run—is always changing. This is particularly true for web application development, where languages, frameworks, and runtimes are all under rapid development.
This rate of change leads to two competing pressures:
Currency: Tools/libraries/runtimes often must be upgraded because of patches to deal with security issues, parts going out of service, changes to minimum requirements in runtimes, or even just new capabilities that are important to the application’s functionality.
Consistency: The bits that make up an application and the environment where it is deployed should be consistent. Deploying a fix should not cause arbitrary changes, and developers should be in control of when tools/libraries/runtimes change, allowing them to update at times that makes sense in the development cycle.
The Continuous Delivery (CD) Pipeline Versioned Base Image (VBI) support was created to help deal with these competing pressures when developing applications for IBM Cloud.
What tools are available in pipeline jobs?
When a Continuous Delivery (CD) pipeline job is run, it executes in an environment defined by a docker image containing a set of tools. These are typically development tools (e.g., javac, node), packaging tools (e.g., npm, maven, and similar), or tools for deploying and managing IBM Cloud services (e.g., ibmcloud, kubectl, helm, cf, etc.).
Previous to the introduction of the Versioned Base Image support, the image used to run pipeline jobs leaned strongly towards consistency over currency. To the greatest extent possible, pipeline jobs ran in an environment with no changes to the tools, so the results were consistent. Unfortunately, over time, the versions of the compilers in the image (for example) became more and more outdated to the point where they no longer matched anything developers were using on their desktop, and in some cases, they had gone out of service. In addition, because all jobs were running on the same image and the IBM Cloud runtime requirements changed over time, sometimes the image had to be updated, occasionally causing users to need to react to these changes after their pipelines stopped working.
The definitive fix for this is to use Custom Images to provide complete control over which tools (and their versions) are being used to build and deploy your applications. Doing this, however, requires a knowledge of Docker that some teams do not have, and it also introduces an ongoing cost in maintaining/updating the custom image.
With Versioned Base Image support, we can provide regular updates to the image that will be used to run pipeline jobs while still allowing you to have control over when your pipelines are affected by those updates.
Versioned Base Image supports multiple image versions
Instead of having a single image, Versioned Base Image (VBI) allows the CD service to maintain a number of curated images. In general, each of these images will contain the most recent, public versions of the tools for deploying to and managing IBM Cloud runtimes, at the time the image was created. This ensures the greatest possible compatibility with IBM Cloud. For the other tools in the image (i.e., development tools, packaging tools, etc.), the most current Long Term Support (LTS) versions are included since these are most often what is required for working in production environments.
Choosing a specific image version for a pipeline (or even for an individual pipeline job) means that image to always be used until a new one is chosen. The implication of this is that since the tools that are included in a particular image are not updated, it will become necessary, over time, to choose a more recent image to maintain compatibility with IBM Cloud runtimes. As such, it is a best practice to move forward to the most recent image as soon as it is available—the pipeline UI displays the version number of the most recent available image to remind you. However, you can decide when to make the move based on your own testing/validation and understanding of the impact.
Image versions are described by tags that follow a loose semantic versioning scheme:
When the number to the left of the decimal changes, the image has been updated significantly in ways that are likely to impact at least some pipeline scripts.
When the number to the right of the decimal changes, it represents an update that is less likely to cause impacts on existing scripts.
In either case, however, there is no guarantee that all possible pipeline jobs will be unaffected by switching to a newer image. It is your responsibility to verify that your pipeline is working as expected after updating to a different version (in the same way that you would verify your local build worked after updating tools in your local development environment).
Using the “latest” image tag
If you are happy living on the bleeding edge, there’s another way you can work. The pipeline can be set to use a distinguished “latest” image tag, which causes the most recent image available to always be used to run your jobs. Since this scenario can potentially require you to react to unexpected changes in the included tools (because a newer image was released), using “latest” is recommended for simple toolchains that are unlikely to be impacted by different tools versions or for development projects that are moving forward rapidly and can deal with the impact of the changes.
Using the Versioned Base Image support
Using the Versioned Base Image support is easy.
Go to the pipeline page for your pipeline.
- Click on the Configure Pipeline menu item:
- Click on the Image version tab:
- To change the image that will be used by default for all jobs in your pipeline, click on the arrow to drop down a list of image versions and select the one you want to use:
It is also possible to configure which image will be used for each job individually. Generally, this should be avoided because it makes the effort of upgrading to a new image painful (since it requires changes to be made in each job), but there may be circumstances where it would be required.
Finding out what tools are in a particular image
The Versioned Base Image documentation lists the tool versions that are available in each image, but you can also generate the list yourself by invoking the default_versions.sh script from within a pipeline job.
Dealing with change
Now that you can run your pipeline jobs with the latest tools for developing and deploying to IBM Cloud, you no longer have to worry about dealing with issues caused by using outdated tools. You can also control when newer versions of tools are introduced into your pipeline so that you no longer have to react to unexpected changes. The best of both worlds!