In the first part of this blog series, we discussed lessons learned from migrating an IBM Cloud Foundry-based Python app to IBM Cloud® Code Engine. The code is part of an IBM Cloud solution tutorial that utilized both Cloud Foundry and IBM Cloud Functions for GitHub traffic data analytics.
In this post, we are detailing how we migrated the Cloud Functions part. How did we address the daily data collection? What Code Engine features were utilized? What are the lessons learned?
For the previous version of our tutorial — “Combining serverless and Cloud Foundry for data retrieval and analytics” (link resides outside ibm.com) — we used a Cloud Functions action to collect traffic data from GitHub. That action was invoked daily by a Cloud Functions alarm trigger based on a defined rule. The action itself (link resides outside ibm.com) was pretty small and written in Python.
Code Engine has a similar concept to triggers and rules — subscriptions allow you to associate an app to an event producer. The ping event has a cron-like syntax and is similar to the alarm trigger in Cloud Functions. Thus, it is a direct replacement and used in our migrated tutorial.
As of this writing, a ping event sends a HTTP POST request to an app and a configured path. An app(lication) is one of two workload types supported by Code Engine. An app serves HTTP requests whereas a job, the second workload type, is executed once and then exits. A job is comparable to the classic batch job.
Although a job comes closer to the daily task of collecting the traffic data, we have to utilize an app. For the migration, we decided to integrate the data collection into the web app and expose it as API function (link resides outside ibm.com)(see below). One reason is that the code to initialize the database connection was already present in the web app and could simply be reused. Even better, we took the advantage and added a manually triggered data collection to the app. With that code change, we only have the Python web app — the additional asset of a cloud function is gone. Thus, only a single code object needs to be built and deployed. The alternative would have been to deploy the data collection as job and invoke it by a dedicated, triggered app (link resides outside ibm.com).
API function called by the Code Engine ping event.
The following command creates the subscription to ping events for the path /collectStats, executed daily at 6 am:
ibmcloud ce subscription ping create --name ghstats-daily --
destination ghstats-app --path /collectStats --schedule '0 6 * * *'
--data '{"token":"SECRET_TOKEN_AS_IDENTIFIER"}'
To secure the API call, a secret token needs to be passed. In addition, we could check the request header for conformance and the correct metadata.
Moving the daily data collection from an IBM Cloud Functions action to an API call in the Python web app deployed to IBM Cloud Code Engine was, more or less, straight-forward. Both Cloud Functions and Code Engine support a cron-like event producer that allows for the scheduling of the daily task.
There were different options on how to port the Cloud Functions action. We decided to integrate it into the web app to simplify code and offer more functionality. For everything in context, read the tutorial on serverless web app and eventing for data retrieval and analytics. The source code is available in the GitHub repository github-traffic-stats (link resides outside ibm.com), the old code version in the branch “cloudfoundry” (link resides outside ibm.com).
We did not discuss how a regular Cloud Functions action could have been migrated to Code Engine. Both serverless compute options support containerized code and, therefore, the deployment vehicle.
If you have feedback, suggestions, or questions about this post, please reach out to me on Twitter (@data_henrik)(link resides outside ibm.com) or LinkedIn (link resides outside ibm.com).