How to run a serverless batch job on your private VM in a VPC with IBM Cloud Code Engine.

IBM Cloud Code Engine has a cool batch feature that allows you to run batch workloads in a serverless fashion. You just have to pay per use, with no cost for the infrastructure.

Still, customers asked whether it would be possible to have the ease of use of Code Engine combined with isolated VMs in the IBM Cloud as execution targets for batch workloads.

What is IBM Cloud Code Engine?

IBM Cloud Code Engine is a fully managed, serverless platform that runs your containerized workloads, including web apps, microservices, event-driven functions and batch jobs with run-to-completion characteristics. The Code Engine experience is designed so that you can focus on writing code and not on the infrastructure that is needed to host it.

What is VPC?

IBM Cloud Virtual Private Cloud (VPC) is a virtual network that is tied to your customer account in IBM Cloud. VPCs offer a “best-of-both-worlds” approach to cloud computing. IBM Cloud VPC gives you the security of a private cloud with the availability, cost-effectiveness and scalability of the public cloud.

Batch workloads on VMs

We published a new variant of running batch workloads in Code Engine. It allows the execution to take place on VSIs (VMs) located in the IBM Cloud Virtual Private Cloud Service (VPC) driven by the Code Engine serverless model.

 This allows you to do the following:

  • Process a large amount of data (e.g., videos, machine learning models, backup files, pay-checks, etc.) that is secured in your private network or on IBM Cloud Object Storage
  • Have complete flexibility over the machine types (e.g., x86, System/Z, GPU)
  • Attach additional storage (e.g., to uncompress .zip files before processing)
  • Run your compute tasks over several hours or even days
  • Run from a single to many thousands of VMs in parallel
  • Connect to any other service running in your VPC or located in your data center

On top of all these benefits, you can have an easy and developer-friendly programming model that abstracts away any details of infrastructure elements and reduce the total cost of ownership for your infrastructure resources by only paying for exactly what you use, without idling resources.

Curious to get hands-on and try it out?

You can run batch workload in your VPC using a VSI proxy — please find the sample in this repository. The readme file has all the details to get started.

Contact IBM Cloud Code Engine Support if you want to learn more or have any feedback.

Learn more about IBM Cloud Code Engine.

Was this article helpful?

More from Cloud

A clear path to value: Overcome challenges on your FinOps journey 

3 min read - In recent years, cloud adoption services have accelerated, with companies increasingly moving from traditional on-premises hosting to public cloud solutions. However, the rise of hybrid and multi-cloud patterns has led to challenges in optimizing value and controlling cloud expenditure, resulting in a shift from capital to operational expenses.   According to a Gartner report, cloud operational expenses are expected to surpass traditional IT spending, reflecting the ongoing transformation in expenditure patterns by 2025. FinOps is an evolving cloud financial management discipline…

IBM Power8 end of service: What are my options?

3 min read - IBM Power8® generation of IBM Power Systems was introduced ten years ago and it is now time to retire that generation. The end-of-service (EoS) support for the entire IBM Power8 server line is scheduled for this year, commencing in March 2024 and concluding in October 2024. EoS dates vary by model: 31 March 2024: maintenance expires for Power Systems S812LC, S822, S822L, 822LC, 824 and 824L. 31 May 2024: maintenance expires for Power Systems S812L, S814 and 822LC. 31 October…

24 IBM offerings winning TrustRadius 2024 Top Rated Awards

2 min read - TrustRadius is a buyer intelligence platform for business technology. Comprehensive product information, in-depth customer insights and peer conversations enable buyers to make confident decisions. “Earning a Top Rated Award means the vendor has excellent customer satisfaction and proven credibility. It’s based entirely on reviews and customer sentiment,” said Becky Susko, TrustRadius, Marketing Program Manager of Awards. Top Rated Awards have to be earned: Gain 10+ new reviews in the past 12 months Earn a trScore of 7.5 or higher from…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters