September 15, 2021 By Robert Walkup
Greg Mewhinney
3 min read

An example of how the IBM Spectrum LSF offering on IBM Cloud can be used for hosting Computational Fluid Dynamics (CFD) application workloads.

The IBM Spectrum LSF offering on IBM Cloud makes it easy to deploy and configure high performance computing (HPC) clusters to host HPC workloads on the virtual private cloud (VPC) infrastructure. In this blog post, we provide a summary of the results from an evaluation of a set of benchmark runs using one of the workloads included in the OpenFOAM toolkit. For more detailed technical information about the evaluation, see the related white paper here.

The OpenFOAM CFD toolkit

OpenFOAM is a widely used open-source toolkit for computational fluid dynamics (CFD). In this post, we provide results from executing the simpleFoam motorbike tutorial, which is included in the toolkit, on IBM Cloud. This test case exercises key iterative solvers in OpenFOAM, including the geometric agglomerated algebraic multigrid (GAMG) solver. The motorbike tutorial also provides a mechanism for exploring OpenFOAM performance over a wide range of grid sizes. 

Environment characteristics

The Spectrum LSF offering on IBM Cloud was used to create all the necessary resources and configure the HPC cluster for evaluating the OpenFOAM workload. The basic elements of the cluster are illustrated in Figure 1. There is a jump host (login system), one or more LSF master nodes, one NFS server node for storage and a static number of LSF worker nodes. Once the base cluster was created, the OpenFOAM software and its dependencies were installed, configured and compiled on the LSF master system. 

We measured the execution time for the simpleFoam motorbike benchmark on two IBM Cloud configurations, each using 128 virtual CPUs on a different number of instances (i.e., worker nodes): (1) a single instance of type cx2-128×256 with shared-memory communication and (2) four instances of type cx2-32×64 with TCP/Ethernet networking between the four instances:

Figure 1. Spectrum LSF cluster components.

simpleFoam motorbike benchmark run results

The figure below shows the simpleFoam processing rate (higher is better) as a function of the size of the grid, where we define processing rate as the number of simpleFoam steps times the number of grid cells divided by the execution time:

Figure 2. The simpleFoam processing rate is shown as a function of grid size for two different IBM Cloud configurations using 128 virtual CPUs: (1) a single instance cx2-128×256 with shared-memory communication and (2) four instances of type cx2-32×64 with TCP/Ethernet interconnect.

For grids larger than ~10M cells, the two different virtual machine configurations provide almost identical performance. For these relatively large grids, communication performance is not very critical, and overall performance is dominated by computation, where bandwidth to memory is a major factor. For grids of ~5M cells or smaller, communication becomes more important, and the single instance cx2-128×256 provides better performance due to more efficient communication via shared memory.

Summary of the OpenFOAM workload evaluation

Good performance was achieved on the simpleFoam motorbike benchmark runs executed in the IBM Cloud environment deployed using the Spectrum LSF offering. Based on these results, we offer a few recommendations for configurations to use depending on the characteristics of the OpenFOAM problem to be solved. 

For OpenFOAM problems with grids of ~10M cells or less, we recommend using the single virtual server instance approach, with one instance of type cx2-128×256. OpenFOAM problems with larger grids may fit on one cx2-128×256 instance, but time to solution will be limited by the number of cores (64) or the number of sockets (4). IBM Cloud offers 64-core, four-socket instance types with far more memory — bx2-128×512 (128 virtual CPUs with 512 GB memory) and mx2-128×1024 (128 virtual CPUs with 1024 GB memory) — and these larger memory instance types can support much larger OpenFOAM problems. However, if time to solution is important, OpenFOAM problems with ~10M grid cells or more can be scaled out to engage more cores, using TCP/Ethernet connected virtual machines. With the current networking infrastructure on IBM Cloud, we recommend limiting scaling to >~ 2*10^5 grid cells per core, or >~ 3*10^6 grid cells per socket. This will ensure that OpenFOAM performance remains in the favorable compute-bound region of the scaling curve. 

Get started with IBM Spectrum LSF on IBM Cloud.

Was this article helpful?

More from Cloud

IBM Cloud Reference Architectures unleashed

2 min read - The ability to onboard workloads to cloud quickly and seamlessly is paramount to accelerate enterprises digital transformation journey. At IBM Cloud, we're thrilled to introduce the IBM Cloud® Reference Architectures designed to empower clients, technical architects, strategists and partners to revolutionize the way businesses harness the power of the cloud. VPC resiliency: Strengthening your foundation Explore the resilience of IBM Cloud Virtual Private Cloud through our comprehensive resources. Dive into our VPC Resiliency white paper, a blueprint for building robust…

Enhance your data security posture with a no-code approach to application-level encryption

4 min read - Data is the lifeblood of every organization. As your organization’s data footprint expands across the clouds and between your own business lines to drive value, it is essential to secure data at all stages of the cloud adoption and throughout the data lifecycle. While there are different mechanisms available to encrypt data throughout its lifecycle (in transit, at rest and in use), application-level encryption (ALE) provides an additional layer of protection by encrypting data at its source. ALE can enhance…

Attention new clients: exciting financial incentives for VMware Cloud Foundation on IBM Cloud

4 min read - New client specials: Get up to 50% off when you commit to a 1- or 3-year term contract on new VCF-as-a-Service offerings, plus an additional value of up to USD 200K in credits through 30 June 2025 when you migrate your VMware workloads to IBM Cloud®.1 Low starting prices: On-demand VCF-as-a-Service deployments begin under USD 200 per month.2 The IBM Cloud benefit: See the potential for a 201%3 return on investment (ROI) over 3 years with reduced downtime, cost and…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters