![Close [x]](http://dw1.s81c.com/i/c.gif){kind=small}
IBM Tivoli® Storage Manager solves storage management needs through a number of easy to manage capabilities. These advanced capabilities include intelligent backup and restore, hierarchical storage management, managed archives, and data redundancy elimination. The storage requirements for both data and metadata can be immense. Using Solid State Drive technology to remove the IO bottleneck of meta-data storage, performance improvements (in some cases up to 229 percent) can be realized for many Tivoli Storage Manager functions. This document provides configuration and tuning for others to experience the gains.
Solid-state drives (SSDs), also called solid-state disks, are persistent block storage devices with no moving parts like a traditional spinning disk (hard disk drive, HDD). Instead, non-volatile SSDs use Flash-based DRAM memory to persist data. As such, they have no moving parts which allow much faster access times by avoiding seek delay. The same interface as HDDs is used for SSDs to allow seamless usage with existing software. SSDs come in HDD-like form factors, as well as PCIe adapters.
IBM Tivoli Storage Manager is an enterprise-wide storage management application. It provides automated storage management services including backup, restore, archive, and retrieve for file servers, database servers, workstations, and personal computers with a variety of operating systems.
The server program provides backup, archive, and space management services to the clients. One can set up multiple servers in an enterprise network to balance storage, processor, and network resources.
The Tivoli Storage Manager server uses a database to track information (for example, meta-data) about server storage, clients, client data, policy, and schedules. In the SSD experiments herein, the server database resides on the SSD.
The server can store and retrieve data using disk and tape storage devices and subsystems from an extensive list of vendors. The server storage media is grouped into storage pools. The storage devices can be connected directly to the server, or connected via local area network (LAN) or storage area network (SAN).
Server administrative interface
The administrative interface allows administrators to control and monitor server activities, define management policies for clients, and set up schedules to provide services to clients at regular intervals. Administrative interfaces available include a command-line administrative client and a web browser interface (called the Administration Center). Tivoli Storage Manager allows you to manage and control multiple servers from a single interface accessible via web browser.
The backup-archive client allows users to maintain backup versions of files, which can be restored if the original files are lost or damaged. Users can also archive files for long-term storage and retrieve the archived files when necessary. The backup/archive client program is installed on file servers, database servers, workstations, and personal computers and is registered with the server.
Application clients allow users to perform online backups of data for applications, such as database programs. The following products provide application clients for use with the Tivoli Storage Manager server: Tivoli Storage Manager for Databases, Tivoli Storage Manager for Enterprise Resource Planning, and Tivoli Storage Manager for Mail.
The following are the details of the system under test (SUT) and test drivers.
Tivoli Storage Manager server, Version 6, Release 2, Level 1
- 4 Intel® Xeon™ E7450 (6-core) processor packages at 2.4 GHz (24 cores total)
- 32 GB real memory
- 2 Intel(R) 82575EB Gigabit Network ports
- 2 Intel(R) PRO/1000 EB Network ports with I/O Acceleration
- 2 IBM 320 GB High IOPS SD Class SSD PCIe Adapters
- 2 Qlogic QLE2460 single-port 4 Gbps fibre HBAs
- 1 Qlogic QLE2560 single-port 8 Gbps fibre HBA (used for tape)
- Microsoft® Windows Server 2008 R2 Enterprise x64 Edition
- Multipath I/O feature installed
- IBM Subsystem Device Driver DSM, Version 2.4.2.1-2
- 8 256 GB disk LUNs on DS8100 model 921 using 8 ranks
- 8 64 GB disk LUNs on DS8100 model 921 using 8 ranks (same ranks as above)
- 1 IBM TS3500 tape library with 4 IBM LTO™ Ultrium® 5 tape drives (shared)
Tivoli Storage Manager clients (32x), Version 6, Release 2, Level 1
- Lenovo ThinkCentre M55 8810-BE2
- 1 Intel Core2 Duo at 2.66 GHz
- 2 GB real memory
- 1 Broadcom NetXtreme Gigabit Network card
- 150 GB IDE disk
- Microsoft Windows XP Professional, SP3
- Netgear® GSM7352S 10 Gbps/1 Gbps 48-port LAN switch
- IBM 2005B32 32-port 4 Gbps fibre switch
The two IBM 320GB High IOPS SD Class SSD PCIe adapters were required Fusion-IO Windows software version 2.1.0. The SSD adapter firmware was updated on all adapters to the 43247 level. The devices were low-level formatted to the default advertised capacity which provides two 160GB devices on each adapter. Finally, NTFS volumes were created using the default allocation unit size. In one set of measurements, a single NTFS volume was used on a single SSD adapter device. In another set of measurements, a RAID5 volume was created using all four SSD adapter devices.
A set of standard file system workloads are used for backup-archive client function performance tests and server function performance tests. The standard file system workloads have the following attributes:
- Randomly assigned file names and directory names using variable length names.
- Maximum directory depth of ten levels.
- Maximum of 32 files per directory.
- Files are created from source data that has a typical compression ratio of three to one when using Tivoli Storage Manager client compression.
- Files effectively contain zero percent duplicate data within each workload volume and between all volumes on all clients.
- Total workload size is scaled depending on the elapsed time of the measured function, the requirements for starting multiple threads, the required measurement repeatability, or the amount of available disk space.
- Workloads are created on newly defined file systems or newly formatted partitions.
Separate workloads are created based on their average file size, 10 KB.
Figure 1. Test environment
All performance measurements in this report were taken in an isolated environment as shown in Figure 1, so there was no resource contention from sources other than the product being measured. The Tivoli Storage Manager server was configured with disk storage containing eight 256 GB Logical Unit Numbers (LUNs) and eight 64 GB LUNs on a DS8100 model 921 using eight ranks of 146 GB 10K RPM fibre channel disk drives using 6+P or 7+P RAID 5 arrays. Each LUN was formatted using the NTFS file system with a 4 KB allocation unit size. The Tivoli Storage Manager server database was located in a single directory on one DS8100 64GB volume, on the SSD RAID 5 volume, or on the SSD JBOD volume. The Tivoli Storage Manager server active log and archive log files were placed on separate DS8100 64GB LUNs.
The Tivoli Storage Manager server database was populated with enough objects and DB2 runstats was executed prior to executing performance measurements so that optimal use of database indexes was achieved. The database automatic reorganization capability was disabled during these tests.
For database backup measurements using a tape device, mount time and volume open time is included in the measurements. IBM LTO™ Ultrium® 5 cartridge tape media and tape hardware compaction was used in these measurements. Refer to the specific measurements table for the tape format. Tape hardware compaction was enabled for all tape measurements in this report.
These measurements are intended to compare performance between the Tivoli Storage Manager server database using a single database directory on a DS8100 volume, a JBOD SSD volume, and a RAID 5 SSD volume. In all of these tests, the Tivoli Storage Manager server active log and archive log directories were placed on separate DS8100 arrays and were not a bottleneck in any of the tests.
Numerous variables and environmental conditions can affect performance. The results presented herein represent one perspective of the performance gains possible in a controlled environment. Others' results may vary.
Workstation client function performance
This section includes measurements for the incremental backup and selective backup client/server functions using workstation clients. All measurements were made with the data flowing over the LAN. The server is configured with four 1GB Ethernet adapters and each client has a single 1 GB Ethernet adapter. These multiple-client tests were balanced so all server LAN connections were equally busy; although, in these tests with small files, the LAN is not a bottleneck. All communications used standard Ethernet frames (1500 bytes).
All client function measurements were made using the command line Backup/Archive client for each client platform.
The incremental backup function is used to back up changed files in the specified client file system. The data is transferred from the client to the server over the LAN and then written to disk. Backup elapsed time depends on the total number of files and directories in the file system; the number of new, changed, or deleted files; and size of the files backed up, as well as other factors. Once the files and directories that need to be backed up have been determined, large file throughput is generally limited by the storage device or network bandwidth, and small file throughput is generally limited by file open/close operations on the client and database processing on the server. Accordingly, the measurements in this section are only done with the 10 KB files workload with focus on the time to determine the changed files in the client file systems and the performance of updating the Tivoli Storage Manager server database with those changes. Tivoli Storage Manager can perform incremental backups using any of several methods optimized for different situations.
Incremental backup using the memoryefficientbackup no option
means the Tivoli Storage Manager client receives a list of files and directories and
their attributes from the Tivoli Storage Manager server for the file system being
backed up, so a comparison can be made to determine which objects are new or changed
and must be backed up and which objects are deleted and must be expired.
Journal-based incremental backup means the Tivoli Storage Manager client maintains its own list of objects that are new or changed and must be backed up and objects that are deleted and must be expired. Because journal-based incremental backup does not query the server for inventory data and does not scan the file system being backed up, it is usually faster than incremental backup using the memoryefficientbackup no option. However, if a large enough number of clients are backing up data at the same time, the Tivoli Storage Manager server database can become a bottleneck. This may result in conditions where journal-based incremental backup is no faster than full incremental backup.
Figure 2. Incremental backup
Table 1. Incremental backup test parameters
| Percent of changed data | 5 |
| Percent of new/deleted data | 0 |
| File storage pool | Yes |
| Compression | No |
| Encryption | No |
| Storage Pool CRC | No |
| Deduplicate | No |
| Storage pool collocation | No |
| Verexists | 1 |
| DatabaseMemPercent | AUTO |
| Txngroupmax | 4096 |
| Txnbytelimit | 25,600 |
| MBs backed up | 16,373 |
| Client system cache | Not purged |
The results of the incremental backup with the various options are shown in Figure 2. Note test parameters are in table 2. Using SSDs yielded a 70 percent throughput increase for incremental backup using non-journal based backup. Journal based backup saw an astounding 229 percent increase using SSDs configured in a RAID 5 array. Having the added benefit of RAID protection did not degrade performance as typically experienced.
LAN selective backup using client deduplication
The selective backup function is used to back up a single copy of every file and directory in the specified client file system. The data is transferred from the client to the server over the LAN and then written to disk. Backup throughput depends on the size of the files processed. Large file throughput is generally limited by the storage device or network bandwidth, and small file throughput is generally limited by file open/close operations on the client and database processing on the server.
Figure 3 shows results for selective backups using client-side deduplication. In the first case (labeled "0,1"), each check for a duplicate chunk finds the chunk is unique, and after the check, the data must be sent to the server. In the second case ("100, 2"), each check for a duplicate chunk finds the chunk is a duplicate, and after the check, the data does not have to be sent to the server; however, the database information for the chunks has to be updated.
The second case also requires the existing version of each file be updated to inactive, and the new version inserted. Essentially, the second case involves more database work which means more stress on the server thus a bigger difference when the database disk is a bottleneck.
Note throughput is based on the amount of data sent to or received from the server. Since tests using deduplication may transfer much less data than tests not using deduplication, the throughput may be much lower even if the elapsed time is equal. For this reason, it is more useful to focus on the differences in elapsed time between these tests than on throughput.
As shown in Figure 3, the elapsed time of first selective backup using client-side deduplication was 20 percent less when using a JBOD SSD volume, and 19 percent less when placing the database directory on a RAID 5 SSD volume compared to using a DS8100 volume. The elapsed time of the second selective backup using client-side deduplication was 57 percent less with the single JBOD SSD volume and 54 percent less when employing a RAID 5 SSD volume compared to the DS8100 volume. The average number of database I/O operations per second (IOPS) executed during the second selective backup tests was 1,200 on the DS8100 volume, 2,900 on the JBOD SSD volume, and 2,660 on the RAID 5 SSD volume. These results show a significant benefit of SSDs for selective backup when there is duplicate data.
Figure 3. Selective backup
Table 2. Selective Backup Test Parameters
| Compression | No |
| Encryption | No |
| Deduplication | Yes |
| Enablededupcache | Yes |
| Storage pool CRC | No |
| Storage pool deduplicate | Yes |
| Storage pool collocation | No |
| DatabaseMemPercent | AUTO |
| Txngroupmax | 4096 |
| Txnbytelimit | 25600 |
This section includes measurements for server functions including inventory expiration, database upgrade, and database backup. The measurements are intended to compare performance between the Tivoli Storage Manager server database using a single database directory on a DS8100 volume, a JBOD SSD volume, and a RAID 5 SSD volume.
The inventory expiration function is used to remove database references to client backup or archive objects from server storage based on Tivoli Storage Manager policy definitions.
The Tivoli Storage Manager server includes the ability to run multiple inventory expiration processes in parallel, which may provide greatly increased throughput as long as the underlying database performance is sufficient. Inventory expiration may require a large amount of random database I/O, and since there is no corresponding network or storage pool data I/O, the performance of inventory expiration is almost completely determined by the meta-data database performance. The database performance depends on the server processor speed, server memory size, database and active log disk subsystem performance, and the organization of data rows within the database pages. Especially important is the disk subsystem performance where using subsystems with write cache enabled can improve database performance.
Figure 4. Inventory expiration
When the DS8100 volume becomes the IOPS bottleneck with ten processes, the inventory expiration throughput was 43 percent faster with the JBOD SSD volume and 44 percent faster on the RAID 5 SSD volume. The average number of IOPS executed during these ten process inventory expiration tests was 1,840 on the DS8100 volume, 3,180 on the JBOD SSD volume, and 3,230 on the RAID5 SSD volume.
Table 3. Inventory expiration test parameters
| Storage pool collocation | No |
| Nodes per client | 1 |
| Backups per node | 3 |
| Verexists | 2 |
| DatabaseMemPercent | AUTO |
| Txngroupmax | 4096 |
| Txnbytelimit | 25600 |
The database upgrade utilities were used to upgrade the Tivoli Storage Manager server database from the native database format used for the Tivoli Storage Manager 5.5 server to the DB2 database used for the Tivoli Storage Manager 6.2 server. This is a lengthy operation because the database formats are completely different. The network method was used for the upgrade because it is the fastest method and does not require intermediate storage resources. For these tests, both the existing database and the new database were on the same system. As a database is moved into the new database structure, the validity of the data is checked against constraints that are enforced in the new database. The upgrade utilities automatically correct some errors in the database.
The upgrade process requires significant processor, memory, and I/O resources. For the same database, the upgrade process will generally require less elapsed time when using faster processors and faster disk storage for the new database. Because the upgrade utilities read data sequentially from the existing server database in page format, the existing database is unlikely to be a bottleneck, and it is not useful to provide faster hardware for the existing database. Although the Tivoli Storage Manager server database consists of more than 100 tables, in practice there are only two or three tables with a large number of database rows; therefore, there are only a few threads that will be executing most of the work. This means that only a few processors are needed and adding more processors will not reduce the elapsed time.
The Tivoli Storage Manager server database upgraded in these tests included information for 128 nodes with multiple incremental backup versions and archives. The total number of stored objects was approximately 60 million, all stored in a random disk storage pool. Backup and archive processes were executed concurrently so the existing database was organized without long sequences of pages allocated to a single table.
Figure 5. Database upgrade
As shown in Figure 5 above, the database upgrade throughput was 100 percent faster when using a JBOD SSD volume and 90 percent faster when using a RAID 5 SSD volume compared to storing the directory on a DS8100 volume. The average number IOPS executed during the upgrade tests was 790 on the DS8100 volume, 1,710 on the JBOD SSD volume, and 1,630 on the RAID 5 SSD volume with an average I/O size of about 30 KB. The single DS8100 volume was a bottleneck for the upgrade operation. While using more than one DS8100 volume would likely improve the database upgrade throughput, neither the JBOD SSD volume nor the RAID 5 SSD volume were a bottleneck.
Table 4. Database upgrade test parameters
| Txnbytelimit | 25600 |
The database backup function was used to make a copy of the Tivoli Storage Manager database in server storage for recovery purposes. In these tests, the Tivoli Storage Manager server database included information for 128 nodes with multiple incremental backup versions and archives. The total number of stored objects was approximately 60 million and all objects were stored in a random disk storage pool. There was no deduplication active on the server for this experiment.
As shown in Figure 6 below, the full database backup throughput was essentially identical for all tests regardless of the placement of the Tivoli Storage Manager server database The IBM High IOPS SSD adapter sequential read throughput was fast enough so that other components were the bottleneck for this operation.
Figure 6. Database backup
Table 5. Database backup test parameters
| DatabaseMemPercent | AUTO |
| Txngroupmax | 4096 |
| Txnbytelimit | 25600 |
The results shown herein display the enormous impact SSDs can have on Tivoli Storage Manager performance when used for the meta-data directory storage compared to a standard HDD disk array. Tivoli Storage Manager must maintain a detailed set of meta-data about the files backed up. This database becomes an IO bottleneck when the queries or updates to the database become significant. SSDs remove the IO bottleneck from the database server thus providing massive throughput gains even in a RAID5 fault tolerant configuration. Incremental backup and selective backup showed gains of up to 229 percent and 50 percent respectively. Inventory expiration and database upgrade experienced significant throughput gains, as well.
Learn
- IBM Solid State storage technology uses a memory-type device for mass storage, rather than spinning disk or tape.
- IBM Solid State Storage
PCIe Adapters are the most advanced NAND clustering technology.
- Tivoli
Storage Manager provides a wide range of storage management capabilities.
- Fusion IO provides solid-state technology
and high-performance I/O solutions.
Discuss
- Participate in the IBM Tivoli Storage
Management forum
Table of contents
Local resources
Tags
Use the slider bar to see more or fewer tags.
For articles in technology zones (such as Java technology, Linux, Open source, XML), Popular tags shows the top tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), Popular tags shows the top tags for just that product zone.
For articles in technology zones (such as Java technology, Linux, Open source, XML), My tags shows your tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), My tags shows your tags for just that product zone.
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
