Calculating throughput for object storage

You can calculate the throughput capability of an object storage system and network for optimal performance.

Before you begin

Use a memory-mapped file system location to provide the source data for benchmarking. If a memory-mapped file system is not possible in your system environment, use subdirectories on the database disks. The preferred method to eliminate bottlenecks on the source disk is to use memory-mapped file system locations, such as the temporary file system (tmpfs) on Linux® operating systems.

Procedure

Use both of the following benchmarking tools to measure throughput capability:
- tsmobjperf.pl Perl script
- SPObjBench.jar Java™ application
To obtain the benchmarking tools, see the Cloud Blueprints . For instructions about using the benchmarking tools, see the Cloud Cache and Object Storage Benchmarking.pdf guide, which is included with the Cloud Blueprints.

Populate a set of 10 1-GB files in a memory-mapped file system. For example, on a Linux system with at least 11 GB of free RAM, issue the following commands:

mkdir /mnt/ramdisk
mount -t tmpfs -o size=11g tmpfs /mnt/ramdisk
for I in `seq 10`; do dd if=/dev/urandom of=/mnt/ramdisk/file.$I bs=1048576 count=1024;
          done

To run a set of automated tests that scales from 1 - 100 data threads, issue the following command:
```
perl tsmobjperf.pl type=type endpoints=endpoint user=”user”
 pass=”pass” bucket=bucket min=1 max=100 step=10
            fslist=comma_delimited_source_files_list
```
where:

type

Specifies the the Simple Storage Service (S3) protocol such as IBM® Cloud Object Storage, Amazon Simple Storage Service (Amazon S3), or other approved S3 protocol object storage systems. Use Azure for Microsoft Azure Blob storage.

endpoints

Specifies a comma-delimited list of one or more IP addresses or URLs for the object storage endpoints. Use the same endpoints as planned for database backup operations to cloud object storage. For Microsoft Azure based systems, specify the URL of a user’s blob storage account.

user

For S3, user specifies a public key ID. For Azure, user specifies the blob storage account name. Enclose the parameter value in double quotation marks.

pass

For S3, pass specifies the secret key for a user who has valid S3 credentials to create buckets and PUT and GET objects in the region that is specified in the endpoint URL. For Azure, the pass value must be a shared access signature (SAS) token with sufficient read/write access rights to the blob storage account over an HTTPS or HTTP connection. Enclose the parameter value in double quotation marks.

bucket

Identifies an S3 bucket, vault name, or an Azure container name for which a credentialed user has PUT and GET access. The bucket name must exist in the object storage system.

min and max

The min and max values specify the minimum and maximum thread counts that are tested.

step

Specifies the increase in thread count from test to test.

fslist

Specifies a comma-delimited list of source files that are used for multipart upload. Use the source files that you created earlier.
Tips:
- Each thread count test uploads 10 1-GB objects per thread. The tool does not remove objects that are created during the test. After the test, you must manually remove objects that are created on the object storage system.
- When you estimate the object storage throughput for the system, use the highest throughput value that was achieved by the automated tests.
- If the throughput estimate is too low to achieve adequate throughput for database backup operations, use more object storage endpoints. Investigate object storage system-related bottlenecks or reconfigure the network to the object storage system. For example, consider a dedicated link for off-premises clouds, use larger capacity Ethernet ports (10 Gb instead of 1 Gb), or use additional bonded ports to achieve greater throughput.