Big Data

4 steps to set up a high-performance computing cloud instance

Share this post:

In recent years, an abundance of data has presented new challenges to companies.

Increasingly, they’ve been turning to high-performance computing (HPC) to tackle these challenges.

HPC is the process of aggregating computing power to accelerate system performance to solve large computation problems in science, engineering or business. The infrastructure for high-performance computing can be costly, so many have decided to host HPC in the cloud.

The steps below show you how to define an HPC cloud instance on SoftLayer, the IBM infrastructure-as-a-service (IaaS) platform.

1. Identify your workload and business requirements.

Before setting up an HPC environment, first consider the workload that you’ll host in the cloud. Then consider the business requirements of the solution. Based on these requirements, the infrastructure team can develop the underlying SoftLayer infrastructure needed to harness the power of HPC in the cloud: a solution that will help the business meet its identified and targeted objectives.

2. Set up the HPC-on-cloud instance.

Now that you’ve identified the technical and non-functional requirements of the workload, it’s time to set up the SoftLayer HPC environment. IBM can assist you with the design and development of your SoftLayer HPC environment, or you can use the SoftLayer self-service portal.

For example, let’s assume the infrastructure team has identified the need for an HPC instance to support its Hadoop workload. Though SoftLayer offers both bare metal and virtual compute nodes, in this case you should select bare metal (dedicated) servers with NVIDIA Tesla GPUs. By using NVIDIA GPUs, the processor of the bare metal server will be able to manage the high compute requirements of the Hadoop workload to accelerate processing performance.

You have the option of hourly or monthly pricing, and you will select the number of bare metal servers you need to meet the operational requirements of the identified workload.

High-Performance Computing cloud 1

3. Configure the servers.

During server configuration and setup, you can select the operating system, memory and storage. These selections should be based on application workload and technical requirements. You can also select the location of the data center where the high-performance computing instance will be hosted. If the workload requires higher throughput with low latency, you should add InfiniBand, which can support up to 56 GB per second of throughput for your HPC on SoftLayer compute instance.

High-Performance Computing Cloud 2

One advantage of the HPC SoftLayer environment is the ability to “auto scale.” This feature allows you to design a Hadoop workload based on steady-state utilization requirements. During peak workload compute utilization, the HPC compute instance will automatically spin up additional compute resources to meet the processing demands of the workload. After utilization has returned to the defined steady state and the “cooldown” period has been met, the system will automatically de-provision the additional resources.

4. Reap the benefits.

By defining your HPC cloud instance on SoftLayer, you receive the benefits of a consumption model that gives you the flexibility to increase or decrease resources based on the changing demands of your business. With SoftLayer, you can take full advantage of the benefits of hosting HPC on the cloud. Check out the post “Empowering high-performance computing in the cloud” to learn more.

To learn more about HPC and other technology and features available with SoftLayer, check out our Cloud How-To webcast series.

More Big Data stories

IBM Aspera powers remote editing for FOX Sports’ coverage of the 2019 FIFA Women’s World Cup

FOX Sports made history at last year’s FIFA Men’s World Cup™ in Russia by using live streaming technology for remote editing on a scale it had never achieved before. In previous years, broadcasters traditionally sent their entire editing staff and equipment overseas to the venue. By taking advantage of  IBM Aspera live streaming technology, FOX […]

Continue reading