Question & Answer
Question
MustGather: What do I collect for support if I am diagnosing a performance problem with poor disk response times and the storage is connected with native Fibre Channel?
Cause
After analyzing Collection Services, Job Watcher, PEX, etc. it is found that jobs are being slowed down by poor disk response times. Collection Services or a rack configuration show that the IOA type is a physical fibre channel adapter, such as a 577D or 577F, and the disk resource type is one of the following:
2145 - IBM SVC or Storwize Family
2107 - IBM DS8000 Family
D910 - EMC
D840 - IBM Flash Systems
Answer
In this configuration, volumes on the external storage are presented to the IBM i host system either with a direct fibre connection or through a SAN switch. Typically, more than one path to each Logical Volume (LUN) is used for redundancy and increased throughput. This document will explain what to collect from the IO Device Driver (IODD) layer in IBM i out to the storage. Layers above the IODD are typically diagnosed using wait bucket analysis available in Collection Services or Job Watcher.
IBM i
IBM i
1. Normal Collection Services, Job Watcher, PEX used to diagnose performance problems. This normally will have been collected during initial PD before it is determined the problem is with the disk subsystem. At a minimum Collection Services will be needed.
Parent Data Collection document for IBM i Performance: http://www-01.ibm.com/support/docview.wss?uid=nas8N1012115
2. PALs and LIC LOGs. This document describes how to collect SYSSNAP which is one method that may be used to collect PALs (with hex and stats data) and VLOGs. Use this command and reference the document for more information on QMGTOOLS and SYSSNAP.
QMGTOOLS/SYSSNAP OUTPUT(*IFS) LICLOGS(Y) PALS(Y) QHST(Y) STRDATE(MMDDYY)http://www-01.ibm.com/support/docview.wss?uid=nas8N1010353
3. Advanced Analysis Macro IODD
Press the Enter key.
Sign in with a service tool profile and password that has authority to Display/Alter/Dump in SST.
Select Option 1 - Start a service tool, and press the Enter key.
Select Option 4 - Display/Alter/Dump, and press the Enter key.
Option 2 - Dump to printer, and press the Enter key.
Select Option 2 - Licensed Internal Code (LIC) data, and press the Enter key.
Select Option 14 - Advanced analysis, and press the Enter key.
Specify option 1 and type iodd, and press the Enter key
On the options screen type dumpalldebug and press the Enter key
Press the Enter key until the spool file is created
Back out of SST using F12
Send the two QPCSMPRT spool files to ECuRep.
The most common switches are Brocade and Cisco. The basic commands are included below, but check with your SAN team or switch documentation if assistance is required.
Brocade - support save
Cisco - show tech-support
It is important to collect snaps or statesaves from the storage system during or as soon as possible after the performance problem has cleared. This will prevent important data from being wrapped out which typically can happen within one to two hours. Contact your Storage Administrator, IBM Storage Support, or Third-party vendor for additional assistance or details.
2107 DS8000 Family: Collect a statesave.
1. Generate a Statesave using DSCLI:* Available in all Releases.
diagsi -action warmstart <Storage Image>
For example, dscli> diagsi -action warmstart IBM.2107-xxxxxxx
Date/Time: May 5, 2021 6:48:46 PM BRT IBM DSCLI Version: 7.7.5.61 DS: IBM.2107-xxxxxxx
CMUD00023W diagsi: Are you sure you want to perform diagnostic control warmstart? [y/n]:y
CMUD00026I diagsi: Diagnostic control warmstart is successfully submitted. It will take some time for the system to complete the request.
-- or --
1. Generate an ODD Dump using DSCLI:
* Available on R7.3+ .
diagsi -action odd <Storage Image>
2. After these have been produced, DS8000 hardware support needs to dial in to the box and offload the files generated.
Contact Flash Systems Support for Details
Related Information
Was this topic helpful?
Document Information
Modified date:
01 March 2022
UID
nas8N1022540