How to to run Continuous Delivery Pipeline stages outside of the IBM public infrastructure using Private Workers.

We are pleased to announce the capability to run Continuous Delivery Pipeline stages outside of the IBM public infrastructure using Private Workers.

Workers are the entities that run the stages of our pipelines, and, up until now, there has been only one means of executing those stages—by using public workers on IBM-managed shared public infrastructure.

With the release of the Private Workers, toolchains can be enhanced with a new private worker tool integration that allows pipeline stages to be configured to run on external Kubernetes environments. Supported platforms include the IBM Cloud Kubernetes Service, IBM Cloud Private, Docker on Desktop, or Red Hat OpenShift.

Why would you want to use private workers?

There are several situations where private workers overcome the constraints of public continuous delivery pipeline workers. Obvious examples are the need for data from a private database or source repository that is not available publicly, the requirement for executing a long-running task, or to adhere to a security policy where jobs need to be run on non-public environments.

In larger organisations, administrators can pre-set private workers on required Kubernetes clusters and make them available for development teams to use.

How do you configure Private Workers?

Private Worker functionality can be typically be enabled in just 30 minutes with the following three steps:

  1. Add the Delivery Pipeline Private Worker tool integration to your toolchain.
  2. Configure your cluster with a private worker.
  3. Choose a private worker in your pipeline stage.

1. Tool integration

A Delivery Pipeline Private Worker tool integration provides the means by which a private worker is made available to pipelines in a toolchain. It provides access to the list of available private workers and to information about them such as status and version number. It also provides the instructions on how to configure a Kubernetes cluster to support private workers.

Create a new tool integration, give it a name, and use the Create button to create a ServiceID and APIKey for yourself. Fill in the name and description on the pop-up dialog and click the Create button. Then, click the Create Integration button to add the private worker tool to your toolchain.

More information on ServiceIDs and API Keys.

2. Cluster configuration

You now need to configure your cluster. Note: This step requires access to your cluster via the CLI to execute some kubectl commands. This needs to be done by someone with admin-level access but it is relatively simple; for the purposes of this post, I used a free IBM Cloud Kubernetes Service cluster to which I have an admin account.

You can click on the Private Worker Tool Integration tile on your Toolchains overview page to get the steps required and simply use the copy button to get the commands to run on your cluster.

Once you’ve done this on the cluster, you can confirm that the new worker is ready to accept requests by going to the Tool Integration > Overview page.

3. Pipeline configuration

We’re almost there. Your new private worker is now available to do some work. You just need to configure a stage in your pipeline to use the worker.

You access this via the new Workers tab on the Stage Configuration page.

All that is left is to run the stage and look at the output in the logs on the Stage History page. For the purposes of this example, I added a simple Test Job to my stage that outputs the environment variables, and the log output shows the worker that ran my job.

So, there you have it. We set-up our toolchain, cluster, and pipeline and successfully ran a stage on that cluster—all outside of the IBM public shared infrastructure—by using the new Delivery Pipeline Private Worker feature. Congratulations!

Watch the demo

Watch this short video for a demo of putting private workers to work on an IBM Cloud Kubernetes Service cluster:

IBM Cloud - Delivery Pipeline Private Workers


IBM Cloud – Delivery Pipeline Private Workers

Contact us

Now you can try it out yourself. For more information, see our documentation. If you have questions, engage our team via Slack by registering here and join the discussion in the #ask-your-question channel on our public Cloud DevOps @ IBM Slack. We’d love to hear your feedback.

More from Cloud

Modernizing child support enforcement with IBM and AWS

7 min read - With 68% of child support enforcement (CSE) systems aging, most state agencies are currently modernizing them or preparing to modernize. More than 20% of families and children are supported by these systems, and with the current constituents of these systems becoming more consumer technology-centric, the use of antiquated technology systems is archaic and unsustainable. At this point, families expect state agencies to have a modern, efficient child support system. The following are some factors driving these states to pursue modernization:…

7 min read

IBM Cloud Databases for Elasticsearch End of Life and pricing changes

2 min read - As part of our partnership with Elastic, IBM is announcing the release of a new version of IBM Cloud Databases for Elasticsearch. We are excited to bring you an enhanced offering of our enterprise-ready, fully managed Elasticsearch. Our partnership with Elastic means that we will be able to offer more, richer functionality and world-class levels of support. The release of version 7.17 of our managed database service will include support for additional functionality, including things like Role Based Access Control…

2 min read

Connected products at the edge

6 min read - There are many overlapping business usage scenarios involving both the disciplines of the Internet of Things (IoT) and edge computing. But there is one very practical and promising use case that has been commonly deployed without many people thinking about it: connected products. This use case involves devices and equipment embedded with sensors, software and connectivity that exchange data with other products, operators or environments in real-time. In this blog post, we will look at the frequently overlooked phenomenon of…

6 min read

SRG Technology drives global software services with IBM Cloud VPC under the hood

4 min read - Headquartered in Ft. Lauderdale, Florida, SRG Technology LLC. (SRGT) is a software development company supporting the education, healthcare and travel industries. Their team creates data systems that deliver the right data in real time to customers around the globe. Whether those customers are medical offices and hospitals, schools or school districts, government agencies, or individual small businesses, SRGT addresses a wide spectrum of software services and technology needs with round-the-clock innovative thinking and fresh approaches to modern data problems. The…

4 min read