Initiating a System Dump Using the HMC

Note: Initiating a system dump will immediately terminate all partitions running on the system. A System dump should be initiated only under the direction of IBM Support.

Preparation
- Initiate a graceful shutdown to all partitions
Before initiating a system dump, attempt to use the partition's native command interface to perform an immediate shutdown on all partitions. For IBM i use a 5250 telnet session, if accessible, to issue PWRDWNSYS OPTION(*IMMED) RESTART(*NO) on each partition. Since the HMC cannot be used to monitor for when the shutdown completes, use an approximate time based on past shutdowns (ping command may be helpful to monitor when tcp shuts down).

To initiate a System dump, do the following:

Step 1: Log on the HMC.

Log on the HMC with a user profile that has the proper task role (hmcsuperadmin, hmcservicerep, or hmcoperator) and resource role (the user must have access to the target system).

Step 2: Click the HMC's 'Open Menu' icon in the upper left, expand Serviceability, select Dumps, and then select the System of interest.

Step 3: After selecting the system, go to the right side Initiate dump selection menu, and then select the Initiate system dump menu item.

Step 4: In the Initiate system dump dialog, read the warning and verify the system name, then mark the checkbox for system dump confirmation. Press Initiate to initiate the dump.

Note that the system dump will immediately terminate all running partitions.

A popup will show up for a few seconds that the dup has been succesful initiated.

Step 5: Wait for the dump to complete.

The server state will change to Error - Dump in progress for several minutes. The server will automatically restart after the dump completes. The state will change to Initializing during the IPL, then Operating/standby. The dump offload will start after the IPL. Depending on the size of the server, the offload can be a long running operation.

Alternatively, the dump progress may be monitor from the HMC Command Line Interface.

Open an HMC command line, and issue the following command:
while true; do ls -ltr /dump/SYSDUMP*; sleep 10; done

It displays all system dump files, with the most recent listed last and update every 10 seconds.
Example:
-rw-rw-r-- 1 ccfw ccfw 18655865 Mar 26 09:23 /dump/SYSDUMP.serial.00000005.20240326083757.gz.IN_PROGRESS

If the last file (most recent) matches the time of your dump, and it shows .IN_PROGRESS on the end, the offload is not yet complete. Once the offload is complete, the .IN_PROGRESS ending is dropped, and the dump can then be transferred.

The offload is completed once the file is named like this
-rw-rw-r-- 1 ccfw ccfw 88754566 Mar 26 10:19 /dump/SYSDUMP.serial.00000005.20240326083757.raw3984596576.gz

Note:
- Do not power off the server. Powering off the server prior to completion of the offload will invalidate the dump, all data will be lost.

Step 7: Send the dump to IBM Support.

If the HMC is configured for remote support, the dump can be sent directly to IBM Support using the Selected button, Call home dump... function. It can also be copied from the HMC and sent to IBM support using http://www.ecurep.ibm.com/app/upload or  https://testcase.boulder.ibm.com. For further information on how to retrieve and send the dump to IBM Support, refer to IBM support document Retrieving an FSP or System Dump Using the HMC

Note: Initiating a system dump will immediately terminate all partitions running on the system. A System dump should be initiated only under the direction of IBM Support.

Preparation
- Initiate a graceful shutdown to all partitions
Before initiating a system dump, attempt to use the partition's native command interface to perform an immediate shutdown on all partitions. For IBM i use a 5250 telnet session, if accessible, to issue PWRDWNSYS OPTION(*IMMED) RESTART(*NO) on each partition. Since the HMC cannot be used to monitor for when the shutdown completes, use an approximate time based on past shutdowns (ping command may be helpful to monitor when tcp shuts down).

- Ensure the HMC "dump fix" is applied
If the HMC is a model 7042-CR9 running V8R8.4 or V8R8.5, ensure the following PTF (or supersede) is applied:
HMC V8R8.4.0 Service Pack 2 with iFix MH01665 or later
HMC V8R8.5.0 Service Pack 1 with iFix MH01663 or later.


To initiate a System dump, do the following:

Step 1: Log on the HMC.

Log on the HMC with a user profile that has the proper task role (hmcsuperadmin, hmcservicerep, or hmcoperator) and resource role (the user must have access to the target system).

Step 2: Open the Manage Dumps Panel for the target system.

a) In the Navigation frame, click the Resources icon, the All Systems.

b) Select the server then View System Properties
Click on the server's name or alternately select the server, then click Actions > View System Properties



c) In the navigation area, click Serviceability > Serviceability.

d) In the Serviceability window, under View and Collect, click Manage Dumps.

Step 3: (optional) Verify the dump parameters.

Click Action > System Dump Parameters and set the system dump parameters as specified by the IBM support representative.

Step 4: Initiate the dump by clicking Action > Initiate System Dump.

Initiate the desired type of dump as directed by an IBM Service representative by clicking the Action button:



Step 5: Read the warning and confirm the dump.

a) Verify the correct server is selected.
This panel allows the user to modify the server so the wrong server could be terminated.

b) Read the warning carefully.
Note that the system dump will immediately terminate all running partitions.

c) If you wish to proceed with the dump click OK to initiate the dump. Otherwise, click cancel to cancel the dump:





Step 6: Wait for the dump to complete.

The server state will change to Error - Dump in progress for several minutes. The server will automatically restart after the dump completes. The state will change to Initializing during the IPL, then Operating/standby. The dump offload will start after the IPL.

Note:
- Do not power off the server. Powering off the server prior to completion of the offload will invalidate the dump, all data will be lost.
- If the server's state is "Incomplete", perform a Operations > Rebuild System to force a state refresh.

After the IPL, the dump offload to the HMC will begin. Depending on the size of the server, the offload can be a long running operation. The Manage Dumps panel shows a list of available dumps (dumps that have been copied to the HMC). Use the Refresh button to check for available dumps.


To view status of the current dump being offloaded, from the Manage Dumps panel, click Status > Dump Offload Progress:



For a dump being offloaded, the dump offload progress will show total size of the dump (source_size) and amount copied to the HMC (size):



Tip: You can monitor the offload progress with a periodic refresh from command line. Example: while true; do ls -ltr /dump/SYSDUMP*; sleep 10; done



Step 7: Send the dump to IBM Support.

If the HMC is configured for remote support, the dump can be sent directly to IBM Support using the Selected button, Call home dump... function. It can also be copied from the HMC and sent to IBM support using http://www.ecurep.ibm.com/app/upload or  https://testcase.boulder.ibm.com. For further information on how to retrieve and send the dump to IBM Support, refer to IBM support document Retrieving an FSP or System Dump Using the HMC

Note: Initiating a system dump will immediately terminate all partitions running on the system. A System dump should be initiated only under the direction of IBM Support.

Preparation
- Initiate a graceful shutdown to all partitions
Before initiating a system dump, attempt to use the partition's native command interface to perform an immediate shutdown on all partitions. For IBM i use a 5250 telnet session, if accessible, to issue PWRDWNSYS OPTION(*IMMED) RESTART(*NO) on each partition. Since the HMC cannot be used to monitor for when the shutdown completes, use an approximate time based on past shutdowns (ping command may be helpful to monitor when tcp shuts down).

- Ensure the HMC "dump fix" is applied
If the HMC is a model 7042-CR9 running V8R8.4 or V8R8.5, ensure the following PTF (or supersede) is applied:
HMC V8R8.4.0 Service Pack 2 with iFix MH01665 or later
HMC V8R8.5.0 Service Pack 1 with iFix MH01663 or later.


To initiate a System dump, do the following:

Step 1: Log on the HMC.

Log on the HMC with a user profile that has the proper task role (hmcsuperadmin, hmcservicerep, or hmcoperator) and resource role (the user must have access to the target system).

Step 2: Open the Manage Dumps Panel for the target system.

In the Navigation frame, expand Systems Management, and then click on Servers. In the Work Area (the "Contents of" frame), select the box next to the target system. On the toolbar, click Tasks > Serviceability > Manage Dumps.

Note: The Service Management, Manage Dumps task cannot be used to dump or set system dump parameters. To start a dump, you must access the Manage Dumps task from the System Management, Servers panel.



Step 3: Verify the dump parameters.

Click Action > System Dump Parameters and set the system dump parameters as specified by the IBM support representative.

Note: In Version 7 and later, system dump parameters can now be set from the HMC (in addition to ASMI). Incorrect values can cause very long dump times.

Step 4: Initiate the dump.

Selec the option to Initiate System Dump under the Action menu button:



Step 5: Read the warning that this dump type will terminate the server and all active partitions, and then confirm the dump.

Read the warning carefully. Note that the system dump will immediately terminate all running partitions. If you wish to proceed with the dump click OK to initiate the dump. Otherwise, click cancel to cancel the dump:



Step 6: Wait for the dump to complete.
Do not power off the server. The server will automatically restart after the dump completes.
Note: Powering off the server prior to completion of the offload will invalidate the dump, all data will be lost.

The Manage Dumps panel shows available dumps. Use the Refresh button to check for available dumps.

To view status of the current dump being offloaded, from the Manage Dumps panel, click Status > Dump Offload Progress:



For a dump being offloaded, the dump offload progress will show total size of the dump (source_size) and amount copied to the HMC (size):



Tip: You can monitor the offload progress with a periodic refresh from command line. Example: while true; do ls -ltr /dump/SYSDUMP*; sleep 10; done



Step 7: Send the dump to IBM Support.

If the HMC is configured for remote support, the dump can be sent directly to IBM Support using the Selected button, Call home dump... function. It can also be copied from the HMC and sent to IBM support using http://www.ecurep.ibm.com/app/upload or ftp://testcase.software.ibm.com/toibm/hw. For further information on how retrieve and send the dump to IBM Support, refer to Rochester Support Center knowledgebase document N1014349, Retrieving an FSP or System Dump Using the HMC (Version 7 and Later). To link to document N1014349 immediately, click here .

Note: On POWER5 servers managed by two HMCs, the dump can be retrieved by either, or both, HMCs. Both HMCs attempt to retrieve the dump. The first HMC to finish reading the dump will delete the dump file from the server. If the dump does not appear on the HMC that initiated the dump, look for another HMC attached to the system. To search for another HMC, expand Systems Management, then Servers. In the work area, select the target system, then click the Actions button, Connections, then Disconnect Another HMC. This will show if another HMC is attached. If an HMC does show in the list of HMCs, locate the HMC and look for the dump on that HMC.