Using API-led connectivity for successful BPMN implementations with Camunda Workflow Engine on IBM Cloud.

Digitalization projects often require a joint effort between developers and subject-matter experts. The success of these projects heavily relies on good communication in the team. Business Process Modeling and Notation (BPMN) comes in handy when complex processes need to be analyzed and improved, and once a process has been represented with a BPMN diagram, a process execution engine (i.e., workflow engine) can be employed to establish the connection with the respective software components.

Camunda Workflow Engine is an open-source process execution engine. BPMN diagrams created with the Camunda Modeler application can easily be deployed on the workflow engine to orchestrate business processes. When used in combination with applications deployed on the IBM Cloud Code Engine platform, Camunda provides a cost-efficient solution for BPMN implementation needs.

This post will explain how to use the Camunda Workflow Engine on IBM Cloud. You will learn how to use Camunda tasks and configure remote applications to get started with your own BPMN. Also, you will be able to start processes using Camunda Workflow Engine’s dashboard, Camunda Modeler or any REST client (e.g., cURL, Postman, etc.).

Step 1: Create resources

For this tutorial, you will need two applications — a Camunda Workflow Engine deployed on a Kubernetes cluster and a sample Python Flask application, as shown on the diagram below:

1. Install IBM Cloud CLI and the required plugins

You can find installation instructions for different platforms in the IBM Cloud CLI documentation. After installing ibmcloud cli, install the container-registry plugin by running the following command:

ibmcloud plugin install container-registry

Install the kubernetes-service plugin by running the following command:

ibmcloud plugin install kubernetes-service

2. Install Docker and Kubernetes

You can find Docker installation instructions for different platforms here.

After installing Docker, please proceed with installation of kubectl. You can find kubectl installation instructions here. 

3. Pull Camunda Workflow Engine

Camunda Workflow Engine is available both on Github and on Dockerhub for download. For the sake of simplicity, we opted to use the Docker image.

After pulling the image, please follow the tutorial on deploying an application to Kubernetes.

We will use a sample application on IBM Cloud.

4. Push the image to your container registry on IBM Cloud

You need to first tag the image according to the namespace (e.g., “camunda”) and region (e.g., “de.icr.io”) of your private container registry on IBM Cloud. Then push the image to this container registry on IBM Cloud, from which it can be pulled by a Kubernetes cluster.

Now, run following command:

ibmcloud login

Please enter your credentials:

ibmcloud cr login

ibmcloud cr namespace-add camunda

docker tag camunda/camunda-bpm-platform:latest de.icr.io/camunda/camundaengine:latest

docker push de.icr.io/camunda/camundaengine:latest

5. Provision a Kubernetes cluster on IBM Cloud

On IBM Cloud’s dashboard, please click on the blue Create Resource button and enter “Kubernetes Service” in the search field. Once you click on the Kubernetes Service button, you will be redirected to a Kubernetes cluster provisioning page. When provisioning your cluster, you can pick the Free Pricing Plan. Take note of the name of your Kubernetes cluster as you will need to enter it in Step 6. 

6. Deploy your application to the Kubernetes cluster

Prepare your deployment YAML file (i.e., named “deployment.yaml”): 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: camunda-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: camunda
  template:
    metadata:
      labels:
        app: camunda
    spec:
      containers:
      - name: camunda-container
        image: de.icr.io/camunda/camundaengine:latest

Make sure you are logged in to IBM Cloud using the ibmcloud cli. Please run following commands using the name of your cluster instead of “mycluster-fra05-u3c.2×4”: 

ibmcloud ks cluster pull-secret apply --cluster mycluster-fra02-u3c.2x4

ibmcloud ks cluster config --cluster mycluster-fra02-u3c.2x4

Now apply the deployment.yaml file you created:

kubectl apply -f deployment.yaml

And as the last step, expose the port 8080:

kubectl expose deployment camunda-deployment --type=NodePort --name=camunda-service --port=8080

Use the Kubernetes dashboard (or Kubernetes commands) to read the external IP address and the node port of your deployment.  You can reach Camunda Cockpit by visiting http://<external-ip-address>:<nodeport>/camunda/app/cockpit/default/#/dashboard.

The default credentials for username and password are “demo”.

7. Deploy a sample Python Flash application on Code Engine

To create a Python Flask application from a template, go to the IBM Cloud dashboard, find the + sign, Expand Global Navigation on the bottom left side and follow App Development > Starter Kits. Choose Python Flask App, specify the application name and resource group in the Create menu and click the blue Create icon.

By clicking Deploy your app in the Deployment Automation card, you can choose the platform that hosts your application. Select Code Engine as deployment platform, enter a valid API key and verify that the Container registry Region, Namespace, Resource group and Project are correct.

 Finally, configure the DevOps toolchain by verifying that the suggested Region is correct. After clicking Create, you are presented again with the Deployment Automation card. This time, it will be populated with the application name that links to a preconfigured toolchain.

If the IBM Cloud where you are provisioning services is a different account than yours (e.g., you are using your company’s IBM Cloud account), to successfully run the pipeline and deploy the application, create and copy an access token on your Git account. From the IBM Cloud dashboard go to Developer tools, click on the toolchain of your application and follow Delivery Pipeline > Environment properties. There you can edit the git-token value and paste your personal Git access token.
The CI/CD pipeline can be triggered manually or automatically by a commit-push to the master branch. Insights on your application (e.g., endpoint, number of instances, CPU and Memory configuration) can be retrieved and modified via the IBM Cloud dashboard > Code Engine.

Step 2: Configure Camunda Workflow Engine

First you need to make sure that you have installed Camunda Modeler (i.e., Desktop Modeler) on your computer. Next, you will need to open the BPMN diagram found in this Git repository in Camunda Modeler. The BPMN diagram looks like the image shown below.

Once you have opened the model, you will need to edit the URL in Connector configuration of “Task2” and “Task3” respectively so that they are pointing to your Python Flask application deployment on IBM Cloud Code Engine.

Afterwards you will need to save your BPMN diagram and deploy it on your Camunda Workflow Engine using the deployment button in Camunda Modeler. In the pop-up window for deployment, you will need to replace localhost with the IP address of your Kubernetes Cluster and the (exposed) port of your Camunda deployment:

Step 3: Run the workflow

There are three options to start a process:

• Use the Camunda Modeler (Version >= 4.5.0) application by clicking on the Play icon.
• Use Camunda Workflow Engine’s dashboard by clicking on the Home icon, selecting Task List and clicking on Start process button.

• Send the respective REST Call from any client application (e.g., curl --location --request POST 'http://<ipAddress>:<port>/engine-rest/process-definition/key/<idOfYourProcess>/start' --header 'Content-Type: application/json' --data-raw '{"businessKey":"default"}' ).

All options allow you to forward variables in the same operation/REST Call to Camunda Workflow Engine, which you can use to manipulate your workflow (i.e., you can send in body of the cURL command '{"businessKey":"default", "variables":{"myvariable1":{"value":"somevalue"}, "myothervariable":{"value":"someothervalue"}}}').

You can also start a message event by sending a REST Call like curl --location --request POST 'http:// ://<ipAddress>:<port>/engine-rest/message' --header 'Content-Type: application/json' --data-raw '{"messageName":"takeoverMessage", "businessKey":"default", "resultEnabled":true, "all":true, "withoutTenantId":true}' while a process is running. This way, you can start the execution of another path (i.e., using {"messageName":"starteventMessage" for Task6) or take over and redirect some tasks from a human task (i.e. ,using {"messageName":"takeoverMessage" for Task8) while a process is still running. Any latency related challenges (e.g., start up time for Code Engine applications) in regards to timeouts are overcome by the fact that the Camunda rest client simply has no timeout for rest calls.

Summary

In this tutorial, you have seen how to deploy a workflow engine on a Kubernetes cluster and deploy an application on IBM Cloud Code Engine.

With the help of a workflow engine, you can now orchestrate your microservices and review them with subject-matter experts easily. Thanks to IBM’s Code Engine platform you can now implement components that are reusable across multiple projects. Since only the workflow engine needs to always reachable, you can take advantage of cost savings by deploying all your microservices through Code Engine. In particular, applications deployed on Code Engine will scale down to zero instances automatically if not needed. Enjoy the cost savings.

If  you should have any questions on the IBM Cloud Kubernetes Service, IBM Cloud Code Engine or Camunda Workflow Engine, feel free to contact barbaros.oezdemir@at.ibm.com, david.alber-cic@ibm.com or helene.hochrieser-cic@ibm.com .