IBM Support

MSD Handling for Very Large Systems

Question & Answer


Question

Over the years, IBM i partitions have increased memory allocations. As the IBM i memory allocations have grown, so too have Main Storage Dump (MSD) sizes. What worked 20 years ago, when the average MSD was less than 1 GB, might not work today (when an MSD can grow to several TB). To keep pace with this growth, IBM added functionality and improved performance when it comes to collecting an MSD. This document explains the available options for IBM technical review of an MSD, when the data to be transferred is very large (100 GB+).

Cause

Growth in size of systems

Answer

The severity of the problem should be taken into account to decide which option, or series of options, is necessary.  A development or test environment might not require immediate, online review.
These options are listed in the order of immediacy.
Option 1:  Review the MSD remotely during copy to ASP:
If the LPAR memory is 500GB or larger, it is recommended to get an IBM technical specialist connected remotely while the MSD is still copying.
At the MSD IPL screens, while the MSD is copying to ASP, the MSD can be reviewed at the console.
Using a screen sharing application, such as Webex or Microsoft Teams, an IBM technical specialist can review the MSD.  The application should be set up to allow the IBM specialist to control the console remotely.
This option is the earliest for MSD review, and provides the opportunity to determine whether the failure is a known problem or if it is new.  If the failure can be uniquely identified, the MSD copy to ASP could be safely abandoned.  It also can avoid the necessity for transfer of the MSD to IBM.  This option does require that an IBM specialist is available to perform the review.
A lower-level connection, direct to Licensed Internal Code, informally referred to as a Piranha connection, is also an option at that point of the MSD IPL and collection.  This option requires a direct IP connection, usually through a support function called IBM Assist On Site (AOS) with port forwarding enabled.  As networks and PCs are increasingly controlled for security reasons, the AOS connection might not be allowed to run or connect to IBM AOS servers.
Option 2:  Review the MSD once the partition is back up or Step Mode IPLed past Storage Management Recovery:
If the partition cannot IPL all the way up to OS without failing, this is likely the best path: IPL in Step Mode past Storage Management Recovery, and then review the MSD at DST with a screen sharing application or with AOS port-forwarded Piranha, as described in Option 1.

If IPL back up the OS succeeds, and the final (compressed) size of the MSD is over 100GB, it may be more efficient to get IBM connected remotely to review the dump.  This can, and should, be done in parallel to sending the MSD to IBM.  Again, screen sharing with IBM Support control or setting up AOS port-forwarded Piranha is needed for this method.
Option 3:  Collect SERVICEDOCS against the MSD and send it to IBM:
SERVICEDOCS is frequently used for diagnosing live problems, such as hanging jobs or looping conditions.  SERVICEDOCS can also be run against an MSD.  The output can give IBM Support and Development an opportunity to review MSD data before getting access directly to the MSD itself.
  • STRSST and sign in to SST
  • Option 1 Start a service tool
  • Option 6 Main storage dump manager
  • Option 2 Work with copies of main storage dumps
  • Find the MSD that just occurred, put a 5 in front of it to Display/Print, and press enter
  • Option 2 for System data
  • Put a 6 (Print) in front of Advanced Analysis and press enter
  • Page down, find SERVICEDOCS, put a 1 in front, and press enter
  • Leave the Options field blank and press enter
NOTE: If the MSD was forced due to a system hang, lock information might be useful. In that case, type –lock in the options field before pressing enter.
These instructions create a spooled file called QPCSMPRT. Send the spooled file, in text file format, into IBM for review.
NOTE: SERVICEDOCS provides a limited view of the MSD. Further data might be requested by IBM, including the MSD itself, IBM i SYSSNAP, LIC logs, or PALs.
IBM recommends performing this option parallel to transferring the MSD to IBM.  SERVICEDOCS against the MSD is beneficial when remote access to the system is not possible or allowed.

Option 4: Sending an MSD in to IBM for review:
 
IBM i QMGTOOLS has MSD copy and transfer functions.  The default transfer to IBM is using an https-based mechanism called IBM Secure Diagnostic Data Upload Utility (IBMSDDUU).  This mechanism automatically breaks the transfer into multiple transfers to optimize network throughput to IBM.  IBM highly recommends obtaining the latest version of QMGTOOLS, whenever it is used.  There is a check for updates option in the QMGTOOLS menu.

NOTE: If you are a Blue Diamond customer, there are special transfer requirements -

[{"Type":"MASTER","Line of Business":{"code":"LOB57","label":"Power"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG60","label":"IBM i"},"ARM Category":[{"code":"a8m0z0000000C4BAAU","label":"IBM i"}],"ARM Case Number":"","Platform":[{"code":"PF012","label":"IBM i"}],"Version":"7.1.0;7.2.0;7.3.0;7.4.0;7.5.0"}]

Document Information

Modified date:
08 March 2023

UID

nas8N1021443