September 15, 2021 By Sophia Wen 2 min read

A look at the auto-scaling capability offered by IBM Spectrum LSF in the HPC cluster solution.

The HPC cluster blog post and tutorial show you how to create an HPC cluster on IBM Cloud. This post is to focus on the auto-scaling capability offered by IBM Spectrum LSF in the HPC cluster solution.

The parameters related to auto-scaling are worker_node_min_count and worker_node_max_count:

Your cluster will keep worker_node_min_count workers regardless of whether there are jobs running in the cluster. When jobs request computing resources more than worker_node_min_count workers, LSF auto-scaling will automatically add more workers — up to worker_node_max_count workers in the cluster — to satisfy jobs’ demands. These additional computing resources (i.e., total number of workers – worker_node_min_count ) will be removed from the cluster after the job demand has diminished and the resources have been idling for 10 minutes.

Updating auto-scaling parameters

After you create an HPC cluster, you can still adjust the auto-scaling parameters. This can be done by changing the LSF configuration files inside the cluster. This page shows you how to change worker_node_max_count. You can also adjust the default 10-minute idling time if you want to keep these additional computing resources created by auto-scaling for a longer time. We would suggest keeping 10 minutes as the minimum value. This timer can be adjusted following the steps here.

One of the nice features of LSF auto-scaling is to allow you to mix different types of workers in one cluster. The instructions to add multiple compute profiles are provided here. You can then select the best choice of computing resources depending on the performance characteristics of your workloads.

Conclusions

This blog post shows you how to configure IBM Spectrum LSF auto-scaling parameters with the HPC cluster solution. Try to create a HPC cluster and see how LSF auto-scaling manages computing resources dynamically. You can learn more about LSF auto-scaling here.

Was this article helpful?
YesNo

More from Cloud

Announcing Dizzion Desktop as a Service for IBM Virtual Private Cloud (VPC)

2 min read - For more than four years, Dizzion and IBM Cloud® have strategically partnered to deliver incredible digital workspace experiences to our clients. We are excited to announce that Dizzion has expanded their Desktop as a Service (DaaS) offering to now support IBM Cloud Virtual Private Cloud (VPC). Powered by Frame, Dizzion’s cloud-native DaaS platform, clients can now deploy their Windows and Linux® virtual desktops and applications on IBM Cloud VPC and enjoy fast, dynamic, infrastructure provisioning and a true consumption-based model.…

Microcontrollers vs. microprocessors: What’s the difference?

6 min read - Microcontroller units (MCUs) and microprocessor units (MPUs) are two kinds of integrated circuits that, while similar in certain ways, are very different in many others. Replacing antiquated multi-component central processing units (CPUs) with separate logic units, these single-chip processors are both extremely valuable in the continued development of computing technology. However, microcontrollers and microprocessors differ significantly in component structure, chip architecture, performance capabilities and application. The key difference between these two units is that microcontrollers combine all the necessary elements…

Seven top central processing unit (CPU) use cases

7 min read - The central processing unit (CPU) is the computer’s brain, assigning and processing tasks and managing essential operational functions. Computers have been so seamlessly integrated with modern life that sometimes we’re not even aware of how many CPUs are in use around the world. It’s a staggering amount—so many CPUs that a conclusive figure can only be approximated. How many CPUs are now in use? It’s been estimated that there may be as many as 200 billion CPU cores (or more)…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters