Enjoy the flexibility and cost of IaaS with the convenience of serverless.

For the last several years, serverless has grown in popularity significantly. The following are some key attributes attracting customers:

  1. Only focus on code, not the infrastructure.
  2. Only pay for what you use/no pay for idle. No capacity management needed.
  3. Transparent and granular scaling.

Gap: Any VM config as serverless runtime

However, specifically for load-intensive or more complex workloads, there is still some unsatisfied demand that is not fully addressed by today’s serverless offerings, such as being too constrained with respect to available CPU, memory and disk capacity. This post intends to describe how we think this could be addressed.

The foundational thought is that in established serverless compute offerings, code gets isolated by running in a container, (micro-)VM, etc. As part of that, there is always a dedicated (but relatively limited) amount of CPU, memory, etc. available per invocation. One would ask themselves why there has to be a limitation to this versus giving the user the broadest possible choice, which allows them to pick the configuration that fits their needs best.

Solution: Drive serverless execution from client-side

As shown in the figure above, the user-facing serverless experience would remain the same as what is available today. However, the user would get the option to select the resources available for execution from the full spectrum of VM sizes available, not just a more constrained subset of CPU, mem, disk and network capacities.

Benefits

An additional benefit of this model is that it would be available at the price point of regular VMs. On the negative side, this approach also means increased invocation latency, which is actually acceptable for many workloads that need many minutes of processing anyway. At the same time, this also means that the scope of management responsibility for the user is higher (e.g., OS patch management would be up to them). Having talked to many potential customers about this, however, they seem to be very willing to accept this trade-off, given the strong overall value proposition.

Serverless invocation up to 1 TB of memory size

Certain workloads (e.g., in the ML/AI or data processing space) can benefit significantly from running as much as possible on the same machine for low-latency, in-memory data sharing. To be concrete, this means up to 128 vCPUs, 1TB of memory, many TBs of local SSD disk, etc. are available per invocation and, potentially, hours of execution time if needed.

Driven by real-world use cases

Serverless purists might be claiming this is deviating from the sub-seconds provisioning times associated with serverless since the very early days. However, it’s important to acknowledge that there are many real-world use cases where there is still significant customer value in exposing a serverless interface. For a given workload that runs many minutes, slightly extended provisioning times are not an issue. Usually, VMs can be provisioned within a very short period of time (seconds), so the delta is not that big.

Significant price advantages

Another important aspect addressed by this solution is that it offers price competitiveness at scale. While there is value in a higher level of management abstraction provided by in-market serverless offerings, when comparing their price with the price of a raw CPU-sec or MB-sec level provided by raw VMs, buying the raw VM infrastructure can more cost-attractive, specifically when running at large scale. Obviously, this also comes with management responsibility for the VM shifting to the customer, but this is often accepted.

It’s also very important to note that this solution offers the same “only-pay-for-what-you’re-running” model. That means there is no capacity pool to manage, which is traditionally either always over- or underprovisioned.

Container images continue to be the packaging format

Also, the approach described in this post allows for the acquisition of infrastructure at the regular VM cost-level. While VMs are provisioned per invocation at the lowest level, the actual code still runs within one container on top of it. This means the underlying container base image can continue to be customized by the user by preinstalling libs and taking other configuration steps.

You’d like all infrastructure you’re using to reside in your account, in your VPC, with support for security groups, public gateways, control over the subnets being used, the ability to use various storage options, large fast local SSDs and many 100’s of GBs of memory.

Available as a simple library

We’re making the capability that provides this abstraction available as part of the Lithops open source project. The extension to this project offers a very simple serverless map/reduce interface, where capacity management is entirely transparent to the user and where all features mentioned above are delivered. This is made available in a very initial version to give you and us, jointly, the opportunity to learn how such a capability should be shaped going forward.

With this library, you can run any kind of map or map reduce operation, while not having to take care of provisioning a pool of VMs, updating apt-get, installing the right version of Python, getting all required libs, copying the respective source code onto these machines, etc. Similar to established serverless services like IBM Cloud Code Engine or IBM Cloud Functions, capacity is always perfectly aligned with the actual demand of running code.

Example

The code snippet below illustrates how this works in a hello-world kind of scenario. When executing this, instead of a container, a VM gets spun up and executes the respective logic:

from lithops import FunctionExecutor

def hello(name):
    return 'Hello {}!'.format(name)

with FunctionExecutor() as fexec:
    fut = fexec.call_async(hello, 'World')
    print(fut.result())

Use IBM Cloud Code Engine wherever possible

How is this related to IBM Cloud Code Engine, our latest GA-level serverless offering?

Wherever IBM Cloud Code Engine is applicable, it should be used as the first choice since it provides the most convenience and quality of service. However, in situations where one or more of the aspects described above are required, this approach should be used as the default fallback solution.

We’d like to express very special thanks to Josep Sampe from URV, who was absolutely instrumental in delivering this capability.

Check out the setup instructions and more detailed documentation.

We’re looking very much forward to your feedback.

Categories

More from Cloud

IBM Cloud inactive identities: Ideas for automated processing

4 min read - Regular cleanup is part of all account administration and security best practices, not just for cloud environments. In our blog post on identifying inactive identities, we looked at the APIs offered by IBM Cloud Identity and Access Management (IAM) and how to utilize them to obtain details on IAM identities and API keys. Some readers provided feedback and asked on how to proceed and act on identified inactive identities. In response, we are going lay out possible steps to take.…

IBM Cloud VMware as a Service introduces multitenant as a new, cost-efficient consumption model

4 min read - Businesses often struggle with ongoing operational needs like monitoring, patching and maintenance of their VMware infrastructure or the added concerns over capacity management. At the same time, cost efficiency and control are very important. Not all workloads have identical needs and different business applications have variable requirements. For example, production applications and regulated workloads may require strong isolation, but development/testing, training environments, disaster recovery sites or other applications may have lower availability requirements or they can be ephemeral in nature,…

IBM accelerates enterprise AI for clients with new capabilities on IBM Z

5 min read - Today, we are excited to unveil a new suite of AI offerings for IBM Z that are designed to help clients improve business outcomes by speeding the implementation of enterprise AI on IBM Z across a wide variety of use cases and industries. We are bringing artificial intelligence (AI) to emerging use cases that our clients (like Swiss insurance provider La Mobilière) have begun exploring, such as enhancing the accuracy of insurance policy recommendations, increasing the accuracy and timeliness of…

IBM NS1 Connect: How IBM is delivering network connectivity with premium DNS offerings

4 min read - For most enterprises, how their users access applications and data is an essential part of doing business, and how they service those application and data responses has a direct correlation to revenue generation.    According to We Are Social’s Digital 2023 Global Overview Report, there are 5.19 billion people around the world using the internet in 2023. There’s an imperative need for businesses to trust their networks to deliver meaningful content to address customer needs.  So how responsive is the…