July 15, 2019 By Jerh O'Connor 4 min read

How to to run Continuous Delivery Pipeline stages outside of the IBM public infrastructure using Private Workers.

We are pleased to announce the capability to run Continuous Delivery Pipeline stages outside of the IBM public infrastructure using Private Workers.

Workers are the entities that run the stages of our pipelines, and, up until now, there has been only one means of executing those stages—by using public workers on IBM-managed shared public infrastructure.

With the release of the Private Workers, toolchains can be enhanced with a new private worker tool integration that allows pipeline stages to be configured to run on external Kubernetes environments. Supported platforms include the IBM Cloud Kubernetes Service, IBM Cloud Private, Docker on Desktop, or Red Hat OpenShift.

Why would you want to use private workers?

There are several situations where private workers overcome the constraints of public continuous delivery pipeline workers. Obvious examples are the need for data from a private database or source repository that is not available publicly, the requirement for executing a long-running task, or to adhere to a security policy where jobs need to be run on non-public environments.

In larger organisations, administrators can pre-set private workers on required Kubernetes clusters and make them available for development teams to use.

How do you configure Private Workers?

Private Worker functionality can be typically be enabled in just 30 minutes with the following three steps:

  1. Add the Delivery Pipeline Private Worker tool integration to your toolchain.
  2. Configure your cluster with a private worker.
  3. Choose a private worker in your pipeline stage.

1. Tool integration

A Delivery Pipeline Private Worker tool integration provides the means by which a private worker is made available to pipelines in a toolchain. It provides access to the list of available private workers and to information about them such as status and version number. It also provides the instructions on how to configure a Kubernetes cluster to support private workers.

Create a new tool integration, give it a name, and use the Create button to create a ServiceID and APIKey for yourself. Fill in the name and description on the pop-up dialog and click the Create button. Then, click the Create Integration button to add the private worker tool to your toolchain.

More information on ServiceIDs and API Keys.

2. Cluster configuration

You now need to configure your cluster. Note: This step requires access to your cluster via the CLI to execute some kubectl commands. This needs to be done by someone with admin-level access but it is relatively simple; for the purposes of this post, I used a free IBM Cloud Kubernetes Service cluster to which I have an admin account.

You can click on the Private Worker Tool Integration tile on your Toolchains overview page to get the steps required and simply use the copy button to get the commands to run on your cluster.

Once you’ve done this on the cluster, you can confirm that the new worker is ready to accept requests by going to the Tool Integration > Overview page.

3. Pipeline configuration

We’re almost there. Your new private worker is now available to do some work. You just need to configure a stage in your pipeline to use the worker.

You access this via the new Workers tab on the Stage Configuration page.

All that is left is to run the stage and look at the output in the logs on the Stage History page. For the purposes of this example, I added a simple Test Job to my stage that outputs the environment variables, and the log output shows the worker that ran my job.

So, there you have it. We set-up our toolchain, cluster, and pipeline and successfully ran a stage on that cluster—all outside of the IBM public shared infrastructure—by using the new Delivery Pipeline Private Worker feature. Congratulations!

Watch the demo

Watch this short video for a demo of putting private workers to work on an IBM Cloud Kubernetes Service cluster:

Contact us

Now you can try it out yourself. For more information, see our documentation. If you have questions, engage our team via Slack by registering here and join the discussion in the #ask-your-question channel on our public Cloud DevOps @ IBM Slack. We’d love to hear your feedback.

Was this article helpful?

More from Cloud

IBM Tech Now: April 8, 2024

< 1 min read - ​Welcome IBM Tech Now, our video web series featuring the latest and greatest news and announcements in the world of technology. Make sure you subscribe to our YouTube channel to be notified every time a new IBM Tech Now video is published. IBM Tech Now: Episode 96 On this episode, we're covering the following topics: IBM Cloud Logs A collaboration with IBM watsonx.ai and Anaconda IBM offerings in the G2 Spring Reports Stay plugged in You can check out the…

The advantages and disadvantages of private cloud 

6 min read - The popularity of private cloud is growing, primarily driven by the need for greater data security. Across industries like education, retail and government, organizations are choosing private cloud settings to conduct business use cases involving workloads with sensitive information and to comply with data privacy and compliance needs. In a report from Technavio (link resides outside ibm.com), the private cloud services market size is estimated to grow at a CAGR of 26.71% between 2023 and 2028, and it is forecast to increase by…

Optimize observability with IBM Cloud Logs to help improve infrastructure and app performance

5 min read - There is a dilemma facing infrastructure and app performance—as workloads generate an expanding amount of observability data, it puts increased pressure on collection tool abilities to process it all. The resulting data stress becomes expensive to manage and makes it harder to obtain actionable insights from the data itself, making it harder to have fast, effective, and cost-efficient performance management. A recent IDC study found that 57% of large enterprises are either collecting too much or too little observability data.…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters