zgetdump - Copy and convert kernel dumps

The zgetdump tool copies a source dump into a target dump with a configurable dump format. The source dump can be located either on a dump device or on a file system. The source dump content is written to standard output, unless you redirect it to a specific file. You can also mount the dump content, print dump information, check whether a stand-alone dump device contains a valid dump tool, or create a non-disruptive kernel dump on a live system.

Before you begin: Mounting is implemented with fuse (file system in user space). Therefore, the fuse kernel module must be loaded before you can use the -m option. Should the fuse module not be loaded, you can load it, for example, with modprobe fuse. You do not need root authority to work with a dump using zgetdump.

zgetdump syntax

Parameters

<dump>

is the file, DASD device or partition, multipath partition of a SCSI disk, NVMe partition, channel-attached tape device, or live system device node where the source dump is located:

Regular dump file (for example /testdir/dump.0)
DASD partition device node (for example /dev/dasdc1)
DASD device node for multi-volume dump (for example /dev/dasdc)
Device mapper multipath partition device node of a SCSI disk (for example /dev/mapper/36005076303ffd40100000000000020c0-part1)
Device node for an NVMe partition (for example, /dev/nvme0n1p1).
Tape device node (for example /dev/ntibm0)
Device node for live system (/dev/mem)

Note: For a DASD multi-volume dump, it is sufficient to specify only one of the multi-volume DASDs as <dump>.

-s <system> or --select <system>

for dumps that capture two systems, selects the system of interest. To check whether a dump contains two systems, use zgetdump -i. The -s option is mandatory when you access the dump of a crashed kdump instance, but returns an error if applied to a regular dump.

A dump can contain data for a crashed production system and for a crashed kdump system. A dump like this is created if a stand-alone dump tool is used to create a dump for a kdump instance that crashed while creating a dump for a previously crashed production system. <system> can be:

prod: to select the data for the crashed production system.
kdump: to select the data for the kdump instance that crashed while creating a dump for the previously crashed production system.

-f <format> or --fmt <format>

uses the specified target dump format <format> when writing or mounting the dump. The following target dump formats are supported:

elf: Executable and Linking Format core dump (default)
s390: S/390® dump

-k <key> or --key <key>

Applies only to the encrypted dumps of KVM guests in IBM® Secure Execution mode, see Working with dumps of KVM guests in IBM Secure Execution mode. <key> specifies a file with the decryption key.

<dump_file>

is the file to which the output is redirected. The default is standard output.

-m <dump> <dir> or --mount <dump> <dir>

mounts the source dump <dump> to mount point <dir> and generates a virtual target dump file instead of writing the content to standard output. The virtual dump file is named <dump>.<FMT>, where <FMT> is the name of the specified dump format (see the --fmt option).

-u <dir> or --umount <dir>

unmounts the dump that is mounted at mount point <dir>. You can specify the dump itself instead of the directory, for example /dev/dasdd1. This option is a wrapper for fusermount -u.

-i <dump> or --info <dump>

displays the dump header information from the dump and performs a validity check.

-d <dumpdevice> or --device <dumpdevice>

checks whether the specified ECKD, FBA, SCSI disk, or NVMe disk device contains a valid dump tool and prints information about it.

<dumpdevice> can be the device node of a DASD device (for example /dev/dasdb), an NVMe disk device node (for example, /dev/nvme0n1), or a multipath device node of a SCSI disk (for example, /dev/mapper/36005076303ffd40100000000000020c0-part1).

-V or --verbose

Shows the detailed memory-map layout when printing the dump header information for s390_ext and ELF dump formats.

-h or --help

displays the help information for the command.

-v or --version

displays the version information for the command.

Using zgetdump to copy a dump

Assuming that the dump is on DASD partition /dev/dasdb1 and that you want to redirect the command output from stdout to a file named dump.elf:

# zgetdump /dev/dasdb1 > dump.elf

The following command writes directly to dump.elf:

# zgetdump /dev/dasdb1 dump.elf

Using zgetdump to transfer a dump with ssh

Assuming that the dump is on DASD partition /dev/dasdd1 and that you want to transfer it to a file on another system with ssh:

# zgetdump /dev/dasdd1  | ssh user@host "cat > dump.elf"

Using zgetdump to transfer a dump with FTP

Assuming that you want to use FTP to transfer a dump to a file, dump.elf, on another system:

Establish an FTP session with the target host and log in.
To transfer a file in binary mode, enter the FTP binary command:
```
ftp> binary
```
To send the dump file to the FTP host issue:
```
ftp> put |"zgetdump /dev/dasdb1" dump.elf
```

Using zgetdump to copy a multi-volume dump

Assuming that the dump is on DASD devices /dev/dasdc and /dev/dasdd spread along partitions /dev/dasdc1 and /dev/dasdd1, and that you want to copy it to a file named dump.elf:

# zgetdump /dev/dasdc > dump.elf

For an example of the output from this command, see Using DASD devices for multi-volume dump.

Using zgetdump to copy a tape dump

Assuming that the tape device is /dev/ntibm0:

# zgetdump /dev/ntimb0 > dump.elf
Format Info:
  Source: s390tape
  Target: elf

Copying dump:
  00000000 / 00001024 MB
  00000171 / 00001024 MB
  00000341 / 00001024 MB
  00000512 / 00001024 MB
  00000683 / 00001024 MB
  00000853 / 00001024 MB
  00001024 / 00001024 MB

Success: Dump has been copied

Using zgetdump to create a dump from a live system

To store an ELF-format dump from a live system in a file called dump.elf issue:

# nice -n -20 zgetdump /dev/crash > dump.elf

For an example of the output from this command, see Creating a kernel dump on a live system.

Checking whether a tape dump is valid, and printing the dump header

Assuming that the tape device is /dev/ntibm0:

# zgetdump -i /dev/ntibm0
Checking tape, this can take a while...
General dump info:
  Dump format........: s390tape
  Version............: 5
  Dump created.......: Tue, 19 Jul 2022 17:26:46 +0200
  Dump ended.........: Tue, 19 Jul 2022 17:27:58 +0200
  Dump CPU ID........: ff00012320948000
  Build arch.........: s390x (64 bit)
  UTS node name......: mylnxsys
  UTS kernel release.: 5.14.21
  UTS kernel version.: #1 SMP #1 SMP Wed May 10 12:18:48 UTC 2023
  System arch........: s390x (64 bit)
  CPU count (online).: 2
  CPU count (real)...: 2
  Dump memory range..: 1024 MB
  Real memory range..: 1024 MB

Memory map:
  0000000000000000 - 000000003fffffff (1024 MB)

Checking whether a DASD dump is valid and printing the dump header

Assuming that the dump is on a partition, part1, of a DASD device /dev/dasdb1:

# zgetdump -i /dev/dasdb1
General dump info:
  Dump format........: s390
  Version............: 5
  Dump created.......: Wed, 20 Jul 2022 11:14:33 +0100
  Dump ended.........: Wed, 20 Jul 2022 11:14:46 +0100
  Dump CPU ID........: ff00012320978000
  UTS node name......: mylnxsys
  UTS kernel release.: 5.14.21
  UTS kernel version.: #1 SMP #1 SMP Wed May 10 12:18:48 UTC 2023
  Build arch.........: s390x (64 bit)
  System arch........: s390x (64 bit)
  CPU count (online).: 3
  CPU count (real)...: 3
  Dump memory range..: 1024 MB
  Real memory range..: 1024 MB
                                                                             
Memory map:                                                                  
  0000000000000000 - 000000003fffffff (1024 MB)

Checking whether a SCSI dump is valid and printing the dump header

Assuming that the dump is on the first partition of a SCSI disk, for example /dev/mapper/36005076303ffd40100000000000020c0-part1:

# zgetdump -i /dev/mapper/36005076303ffd40100000000000020c0-part1
General dump info:                                      
  Dump format........: elf                              
  Version............: 1                                
  UTS node name......: r3545010                         
  UTS kernel release.: 5.14.21
  UTS kernel version.: #1 SMP #1 SMP Wed May 10 12:18:48 UTC 2023
  System arch........: s390x (64 bit)                   
  CPU count (online).: 3                                
  Dump memory range..: 1024 MB                          
                                                        
Memory map:                                             
  0000000000000000 - 000000003fffffff (1024 MB)

Checking whether an NVMe dump is valid and printing the dump header

Assuming that the dump is on the first partition on name space 1 of an NVMe disk, for example /dev/nvme0n1p1:

# zgetdump -i /dev/nvme0n1p1
General dump info:
  Dump format........: elf
  Version............: 1
  UTS node name......: t83lp49
  UTS kernel release.: 5.14.21
  UTS kernel version.: #1 SMP Wed May 10 12:18:48 UTC 2023
  System arch........: s390x (64 bit)
  CPU count (online).: 64
  Dump memory range..: 8192 MB

Memory map:
  0000000000000000 - 00000001ffffffff (8192 MB)

Checking whether a DASD device contains a valid dump record

Checking DASD device /dev/dasdb, which is a valid dump device:

# zgetdump -d /dev/dasdb
Dump device info:
  Dump tool.........: Single-volume DASD dump tool
  Version...........: 2
  Architecture......: s390x (64 bit)
  DASD type.........: ECKD
  Dump size limit...: none

Checking DASD device /dev/dasdc, which is not a valid dump device:

# zgetdump -d /dev/dasdc
zgetdump: No dump tool found on "/dev/dasdc"

Checking whether a SCSI disk contains a valid dump record

Checking SCSI multipath device /dev/mapper/36005076303ffd40100000000000020c0, which is a valid dump device:

# zgetdump -d /dev/mapper/36005076303ffd40100000000000020c0
Dump device info:
Dump tool.........: Single-volume SCSI dump tool
Version...........: 1
Architecture......: s390x (64 bit)

Partition info:
Partition number..: 1
Maximum dump size.: 20473 MB

Checking SCSI multipath device /dev/mapper/36005076307ffc5e300000000000084cf, which is not a valid dump device:

# zgetdump -d /dev/mapper/36005076307ffc5e300000000000084cf
zgetdump: No dump tool found on "/dev/mapper/36005076307ffc5e300000000000084cf"

Checking whether an NVMe disk contains a valid dump record

Checking the first partition of name space 1 of an NVMe disk /dev/nvme0n1, which is a valid dump device:

# zgetdump -d /dev/nvme0n1
Dump device info:
  Dump tool.........: Next Generation (NGDump) dump tool
  Version...........: 1
  Architecture......: s390x (64 bit)

Partition info:
  Partition number..: 1

Checking name space two of the NVMe disk, which is not a valid dump device:

# zgetdump -d /dev/nvme0n2
zgetdump: No dump tool found on "/dev/nvme0n2"

Using the mount option

Mounting is useful for multi-volume DASD dumps. After a multi-volume dump has been mounted, it is shown as a single dump file that can be accessed directly with dump processing tools such as crash.

The following example mounts a multi-volume source DASD dump as an ELF dump, processes it with crash, and unmounts it with zgetdump:

# zgetdump -m /dev/dasdx /dumps
# crash  /dumps/dump.elf /boot/vmlinux-5.14.<xx>-<y>-default.gz
# zgetdump -u /dumps

Mounting can also be useful when you want to process the dump with a tool that cannot read the original dump format. Use the --fmt option to mount the dump with a format other then the default format.

Selecting data from a dump that includes a crashed kdump

The following example mounts dump data for a crashed production system from a DASD backup dump for a failed kdump (see Failure recovery and backup tools for details).

# zgetdump -s prod -m /dev/dasdb1 /mnt

Checking whether a dump has captured two systems

A dump can contain data from two systems. To check for this use zgetdump -i, for example, assuming that the previous dump example contains both a dump from the production system and a kdump kernel dump:

# zgetdump -i /dev/mapper/36005076303ffd40100000000000020c0-part1
zgetdump: The dump contains "kdump" and "production system"
          Access "production system" with "-s prod"
          Access "kdump" with "-s kdump"
          Send both dumps to your service organization

Working with dumps of KVM guests in IBM Secure Execution mode

Secure images can be built to support encrypted dumps with virsh dump. A cryptographic key, the customer communication key, must then be specified with the command that builds the image (genprotimg). You need this key to read the dump of a secure guest. For more information, see Using virsh dump to create dumps on KVM hosts.

In the following examples, se-dump.elf is a file with an encrypted dump and cck is the file with the required key. Both files are in the current working directory.

Use the zgetdump -i option to confirm that se-dump.elf is a valid dump.

# zgetdump -i se-dump.elf 
General dump info:
  Dump format........: ELF (protected virtualization dump)
  System arch........: s390x (64 bit)

zgetdump: No key for dump decryption provided.
Try 'zgetdump --help' for more information.

The (protected virtualization dump) in the output confirms that the dump is for a KVM guest in IBM Secure Execution mode.

Specify the key to display more information.

# zgetdump -i se-dump.elf --key cck
General dump info:
  Dump format........: ELF (protected virtualization dump)
  Version............: 1
  UTS node name......: lnxlp05
  UTS kernel release.: 5.14.21
  UTS kernel version.: #1 SMP Wed May 10 12:18:48 UTC 2023
  System arch........: s390x (64 bit)
  CPU count (online).: 2
  Dump memory range..: 1024 MB

Memory map:
  0000000000000000 - 000000003fffffff (1024 MB)

You can create a decrypted copy of the dump file, and then proceed working with the decrypted version.
```
# zgetdump se-dump.elf se-dump.decrypted.elf --key cck
```
Protect the decrypted dump from unauthorized access.
As an alternative to creating a decrypted dump file, you can mount the dump. A mounted dump is decrypted incrementally, as it is accessed. Especially for large dumps, this method can be advantageous over first creating a decrypted copy of the dump file. In the example, the mount point is a directory /dumps in the file system.
```
# zgetdump -m se-dump.elf /dumps --key cck
```
Analyze the mounted dump with crash, as usual.
```
# crash vmlinux /dumps/dump.elf
```
Unmount the dump when you have completed your work.
```
# zgetdump -u /dumps
```