Performance Tuning Checklist
This checklist is a good place to start when looking at IBM Spectrum Scale performance. It contains a list of "areas to consider" when determining the performance of a IBM Spectrum Scale cluster. Tuning these parameters, in many cases, can greatly improve application performance. This page is intended to remain a list for easy reference. Further details on these or other parameters or tuning tips refer to Tuning Parameters.
|
Getting Started
The biggest challenge when tuning IBM Spectrum Scale is developing a clear understanding of your goals. Before embarking on a performance tuning exercise it is a good idea to clearly define a performance target. Developing a performance target Developing a clear performance target allows you to keep your tuning effort focused. Since tuning is often a game of give and take a focused approach can help you make those value decisions as they arise. A performance target can take many forms:
Whatever your target there are certain characteristics the goal should have.
|
IBM Spectrum Scale Configuration Parameters
Start tuning by verifying these parameters are correct. So what is correct? That is hard to say without knowing the workload and infrastructure. Hopefully the descriptions of these parameters can help you determine the correct value. |
- ignorePrefetchLUNCount
- pagepool
- maxMBpS
- maxFilesToCache
- maxStatCache
- maxBufferDescs
- prefetchThreads
- workerThreads (Pre 4.2 worker1threads)
File System Configuration
- -B Blocksize
- -n Number of nodes
Storage Configuration
- IO Balance
- RAID Level
- Cache Configuration
Scenarios
A single client is not getting expected sequential throughput
If you are not getting the sequential performance from a client you expect check these things.
- Capabilities of the storage
- Network throughput - example nsdperf
- Is the client driving the storage hard enough? Check with "mmdiag --iohist" or "mmdiag --waiters" Modify with: maxMBpS, ignorePrefetchLUNCount
Sample Configurations
This section contains sample configurations with some discussion around why the specific parameters were chosen. Of course it is always recommended that you test any configuration parameters before implementing them on a production cluster |
General use HPC Cluster
A general use HPC cluster is typical at many universities. These clusters need to support a variety of I/O workloads so you typically cannot tune to a specific type of I/O pattern. Even with unpredictable IO workloads these clusters have many properties in common. For example, there are usually three types of server usage:
These recommendations make many assumptions about server memory and communication infrastructure. Remember these are just examples and are a good place to start though you need to adjust as necessary to fit your environment. There are slightly different starting points for the three server types NSD Servers pagepool = 16GB or more - NSD servers do not require large amounts of memory. For more details see NSD Server Tuning User Login Nodes pagepool = 16GB Compute Nodes pagepool = 4GiB |
General use small Direct Attached Cluster
In this configuration you have a small cluster 2-10 nodes that all have direct access to a common set of LUNS over a SAN, for example. All of these nodes are running a variety of applications. All Nodes pagepool = 8GB |