Scenario: Improving performance of migration operations

Edit online

This scenario illustrates how to interpret server monitoring data to improve migration operations.

Problem

Kate is an IBM Storage Protect administrator, and she noticed that the disk-to-tape storage pool migrations run slowly on her Windows server.

Goal

Kate expects the write time to be a value close to the capabilities of the drive.

Data collection

Kate collects the server monitoring data, which is automatically collected by the servermon component.

Analysis and determination of the bottleneck

During the migration operation, data is read from the disk and written to tape. The data is transferred from the disk through a host bus adapter (HBA) to the IBM Storage Protect server. The data is transferred from the server through a separate HBA to the tape system.

Figure 1. Data flow for migration operations
The data flow shows the transfer of data from the disk through a host bus adapter (HBA) that connects the disk to the IBM Storage Protect server. The server writes the data to the tape drives and tape library through a different HBA device.

Following the flow of data, Kate looks at the disk read activity first.

As Kate investigates, she notices that all tape operations are slow, including client backups, reclamation, and database backups to tape. Kate suspects a problem with the tape system, which might be causing slow write times. The tape system is an LTO library with SCSI-attached tape drives.

To determine where the slowdown occurs, Kate runs a disk-to-tape storage pool migration test and collects server instrumentation data, by using the data that is automatically collected in the servermon component. She locates the files that were collected by the servermon component, and finds the files that are time-stamped during a time when migration was slow. She looks for the files that are in the form: YYYYMMDD-HHMM-YYYYMMDD-HHMM-instr.txt, where the time stamps represent the start and end times when the output was gathered.

Kate reviews the file, and finds Thread 61, which processed a total of 966912 KB. Then, she finds the corresponding thread that has a Tape Write category in it, which processed about the same amount of data. She found Thread 34, which processed 968192 KB.

Thread 61 DfMigrationThread (Win Thread ID 4436)  17:39:076-->17:47:38

Operation       Count  Tottime  Avgtime  Min-   Max-    Inst     Total
                                         time   time    Tput     KB
---------------------------------------------------------------------
Disk Read        3777   22.680    0.006  0.000  0.031  42632.8  966912
Thread Wait      3778  487.450    0.129  0.016  0.313
Unknown                  0.061
----------------------------------------------------------------------
Total                  510.191                          1895.2  966912

Thread 34 AgentThread (Win Thread ID 5340)  17:39:07.816-->17:47:38.007

Operation       Count  Tottime  Avgtime  Min-   Max-    Inst     Total
                                         time   time    Tput     KB
---------------------------------------------------------------------
Tape Write      30257  508.816    0.017  0.000  0.141   1902.8  968192
Tape Data Copy  31661    0.863    0.000  0.000  0.016
Thread Wait      3777    0.220    0.000  0.000  0.016
Unknown                  0.292
----------------------------------------------------------------------
Total                  510.191                          1897.7  968192
Kate used the server instrumentation categories in Server instrumentation categories to understand the meaning of the Disk Read, Tape Write, and Thread Wait categories. The server instrumentation output shows the following characteristics:
  • The Thread 61 data shows a high value for the Thread Wait field (487.450 seconds), and low value for the Disk Read field (22.680 seconds). This result indicates that Thread 61 is not a concern because the instantaneous throughput is acceptable and the thread was in a waiting phase. The disk is not a problem.
    Tip: The instantaneous throughput, Inst Tput, is calculated by dividing the Total KB value by Tottime value.
  • The Thread 34 data shows that there is a low value for the Thread Wait field (0.220 seconds), and a high value for the Tape Write field (508.816 seconds). The output also shows that the Inst Tput value for the tape write is very slow (1902.8 KB per second). She concluded that the problem is related to the tape system, as shown by the high Tape Write value.

Problem resolution

Kate investigates the following probable sources of the tape system problem:
  • The tape attachment path
  • The device driver level of the tape drive
  • The SCSI adapter driver level
  • The SCSI adapter settings
After the investigation, Kate upgrades the SCSI adapter device driver. The disk-to-tape storage pool migration improves to 75% of the native capacity. Client backups to tape are also much faster.
Tip: All server speeds depend on the hardware that is used. The values in this scenario might not apply to your system. Refer to the performance characteristics of your tape drive or disk system to determine an acceptable level of performance.