Performance Tuning Checklist

This page has not been liked. Updated 10/20/15, 10:27 PM by ScottGPFSTags: None

This checklist is a good place to start when looking at IBM Spectrum Scale performance. It contains a list of "areas to consider" when determining the performance of a IBM Spectrum Scale cluster. Tuning these parameters, in many cases, can greatly improve application performance.

This page is intended to remain a list for easy reference. Further details on these or other parameters or tuning tips refer to Tuning Parameters.

Checklist topics

Sample Configurations

  • General use HPC Cluster
  • General use small Direct Attached Cluster

Getting started



The biggest challenge when tuning IBM Spectrum Scale is developing a clear understanding of your goals. Before embarking on a performance tuning exercise it is a good idea to clearly define a performance target.

Developing a performance target

Developing a clear performance target allows you to keep your tuning effort focused. Since tuning is often a game of give and take a focused approach can help you make those value decisions as they arise. A performance target can take many forms:

  • A throughput target - " I need to achieve 5 GiB/sec using myapp"
  • An IOP target - " I need to be able to create 1,000 files/second using filecreate.bash"
  • An application transaction - " I need to complete 100 cube requests a minute"
  • A backup or recovery target - " I need to be able to perform a differential backup on 10 million files in 2 hours with a 10% daily change rate and an average file size of 1MB"

Whatever your target there are certain characteristics the goal should have.

  • Measurable - the target should be a metric like "files created per second" that can be easily determined.
  • Tools - The tools used to execute and measure the test should be clearly defined. You should research your application requirement and find or develop a relevant tool to generate a workload. For the most efficient analysis the results of a test run should be readily apparent and not require extensive post processing before they can be interpreted.
  • Relevant - The test workload should be as relevant to the production application as possible. This is the #1 issue with most performance tuning exercises. Many performance tuning engagements fail because the workload is chosen because it is handy, and not necessarily relevant. For example, IOR is a great test of throughput but does not help if you are tuning a file system to provide SMTP email service.

IBM Spectrum Scale Configuration Parameters



File System Configuration


-B Blocksize

-S atime updates

-E mtime updates

Storage Configuration


IO Balance

RAID Level

Cache Configuration


Sample Configurations



This section contains sample configurations with some discussion around why the specific parameters were chosen. Of course it is always recommended that you test any configuration parameters before implementing them on a production cluster


General use HPC Cluster



A general use HPC cluster is typical at many universities, for example. These clusters need to support a variety of I/O workloads so you typically cannot tune to a specific type of I/O pattern. Even with unpredictable IO workloads these clusters have many properties in common. For example, there are usually three types of server usage:

  1. IBM Spectrum Scale NSD Servers
  2. User login nodes
  3. Compute Nodes

These recommendations make many assumptions about server memory and communication infrastructure. Remember these are just examples and are a good place to start though you need to adjust as necessary to fit your environment.

There are slightly different starting points for the three server types

NSD Servers

The assumption here is that these NSD servers are serving the rest of the cluster only. In this configuration you need to account for NSD server IO.

pagepool = 8GB or more - NSD servers do not require large amounts of memory.

maxMBpS = 4000

For more details see NSD Server Tuning

User Login Nodes

The assumption here is that these servers are user interactive nodes. The IO throughput requirements may or may not be high but there are many users doing "ls" operations.

pagepool = 8GB

maxMBpS = 5000

maxFilesToCache = 100,000

Compute Nodes

The assumption here is that these servers are used by applications and that memory is best used by the application and not IBM Spectrum Scale cache.

pagepool = 1GiB

maxMBpS = 5000

maxFilesToCache = default


General use small Direct Attached Cluster



In this configuration you have small cluster 2-10 nodes that all have direct access to a common set of LUNS over a SAN, for example. All of these nodes are running a variety of applications.

All Nodes

pagepool = 8GB

maxMBpS = 5000

maxFilesToCache = 250,000