IBM Support

MustGather: Reporting General System Performance Issues

Troubleshooting


Problem

Follow this document to report a performance issue with your IBM i system.  The document covers the following:  performance questionnaire, collecting and uploading data.

Diagnosing The Problem

  1. QMGTOOLS/CMPVER then F6=Download and update
  2. CALL QSYS/QCMD
  3. QMGTOOLS/SYSSNAP OUTPUT(*IFS) COLLECTDFT(Y) COLSQL(Y) COLPCSNAP(Y) CSDATA(Y) DAYSPRV(3) LICLOGS(Y) PALS(Y)
  4. Attach the .zip file to the case
For full instructions, see Resolving the Problem section.

Resolving The Problem

NOTE: The data cannot be reviewed without a clear explanation of the problem. 
A problem description of "everything is slow", lacks important details; use Step 1 to provide the required details.  

Step 1:  Required Performance Questionnaire 

  1. What needs to be fixed? For example, our payroll jobs, named ABCCO, are running one hour longer then expected.
  2. What business function is impacted by the poor performance? For example, when payroll jobs do not complete by midnight, a financial impact occurs as the business is fined for missing service level agreements.
  3. To what extent has the performance changed and how is the slowness observed or measured? For example, payroll used to complete in few minutes. After an IPL, now the job takes one hour.
  4. What are the IBM i jobs (users or subsystems) responsible for processing the slow function? If you are not sure, what other information can you give us to identify the jobs in question?* For example, DRDA application is slow so focus on the QRWTSRVR jobs with user ID XYZ.
  5. What is the timeline of this issue? For example, we first noticed the problem on Sept 15th at 23:00, but the problem does not happen every night.  On Sept 16th, the job completed normally We need the problem resolved by Sept. 30th.
*If the issue appears to be system wide, pick an example of one important job or workload that is measurably slower and provide us with the details as a focus point.

Step 2: Verify the IBM MustGather tool is installed and current

QMGTOOLS is a light-weight, noninvasive data collector tool provided by IBM.  The tool collects many logs over three days and can take 30 - 60 minutes to complete.
  1. CALL PGM(QMGTOOLS/QGETVER) to display the build version. If it is more than three months old, proceed with step 2.
  2. QMGTOOLS/CMPVER then F6 = download and update
  3. If the QMGTOOLS library does not exist,  install the tool by using these text and video based instructions.

Step 3:  Collect data

While the SYSSNAP command is running, complete the performance questionnaire in Step 1.
  1. CALL QSYS/QCMD
  2. QMGTOOLS/SYSSNAP OUTPUT(*IFS) COLLECTDFT(Y) COLSQL(Y) COLPCSNAP(Y) CSDATA(Y) DAYSPRV(3) LICLOGS(Y) PALS(Y) 
  3. If the problem happened more than three days ago, add STRDATE(MMDDYY) ENDDATE(MMDDYY) to the command in step 2 where MMDDYY is the month, day, year the problem occurred on.

Step 4: If the poor performing job is currently running, do the following

  1. Use WRKACTJOB to locate the job number and user name of a job that is executing poorly.
  2. From WRKACTJOB, use option 5, option 11 to collect a screen capture of the call stack, if possible.
    1. Using the job name, user and number of the poor performing job, run the following SQL statements:
          create table LIBRARY/TSxxxxxxAQ as
            (SELECT *   FROM TABLE(QSYS2.ACTIVE_QUERY_INFO(     
            JOB_NAME => 'JobName',       
            JOB_USER=> 'User',          
            JOB_NUMBER=>nnnnnn))) with data;
 
          create table LIBRARY/TSxxxxxxAJ as
            (SELECT * FROM TABLE (QSYS2.ACTIVE_JOB_INFO
            (DETAILED_INFO=>'ALL',     
            JOB_NAME_FILTER=>'JobName'))                              
            WHERE JOB_NAME = 'nnnnnn/User/JobName') with data

Step 5:  Instructions for Sending Data to IBM i Support

NOTE: If you are a Blue Diamond client, use this link for instructions on sending the data.
  1. The data is stored in the IFS under the /tmp/SystemName_SYSSNAP_timestamp.zip directory. 
  2. If you collected data in Step 4 then do the following:
    •      CRTSAVF LIBRARY/IBMDATA
           SAVOBJ OBJ(TSXXXXXXAQ TSXXXXXXAJ) LIB(LIBRARY) DEV(*SAVF) SAVF(LIBRARY/IBMDATA)
  3. NOTE: you need to set up a transfer id (and optionally store it in QMGTOOLS) to transfer the data via QMGTOOLS. 
  4. Use QMGTOOLS/FTP2IBMCMD to send the data. 
    • QMGTOOLS/FTP2IBMCMD FTPTYPE(*HTTPS) IBMID(transferID) IBMPWD(transferPWD) INPUT(*IFS) CASENBR(TSxxxxxxxxx) IFSFILE('/tmp/SYSNAME_SYSSNAP_202210272037.zip')
    • QMGTOOLS/FTP2IBMCMD FTPTYPE(*HTTPS) IBMID(transferID) IBMPWD(transferPWD) INPUT(*IFS) CASENBR(TSxxxxxxxxx) SAVEFILE(LIBRARY/IBMDATA)
  5. If you cannot use QMGTOOLS/FTP2IBMCMD to FTP out of the system, see alternative methods.
                                       

[{"Line of Business":{"code":"LOB68","label":"Power HW"},"Business Unit":{"code":"BU070","label":"IBM Infrastructure"},"Product":{"code":"SWG60","label":"IBM i"},"ARM Category":[{"code":"a8m0z0000000CI9AAM","label":"Performance-\u003EMustGather"}],"Platform":[{"code":"PF012","label":"IBM i"}],"Version":"All Versions","Type":"MASTER"}]

Historical Number

Product Synonym

Document Information

Modified date:
16 December 2024

UID

ibm16574525

Page Feedback