IBM Support

Multi-threading in makedumpfile might cause corrupt vmcores

Flashes (Alerts)


Abstract

On Red Hat Enterprise Linux (RHEL) 10.0 onwards, the makedumpfile has multi-threading enabled by default to speed up the vmcore collection processes. However, due to bugs in makedumpfile, the multi-threading logic has data races, which can sometimes cause some parts of the generated vmcore to become corrupt.

This bug is more reproducible on systems with larger memory, such as 64 TB RAM, but can also be reproduced anywhere from 2% - 20% of vmcores on a smaller system with 128 GB RAM.

Content

Linux Releases Affected
Red Hat Enterprise Linux (RHEL) 10.0
 
IBM Systems Affected
All IBM Power Systems
 
Symptoms
If you use crash-utility with a corrupted vmcore (due to this issue), the utility shows the following failed: unable to compress error messages: 
	crash: compressed kdump: uncompress failed: 0
	crash: read error: kernel virtual address: c0001e2d2fe48000  type: "hardirq thread_union"
	crash: cannot read hardirq_ctx[930] at c0001e2d2fe48000
	crash: compressed kdump: uncompress failed: 0
	crash: read error: kernel virtual address: c0001e2d2fe40000  type: "hardirq thread_union"
	crash: cannot read hardirq_ctx[931] at c0001e2d2fe40000
	crash: compressed kdump: uncompress failed: 0
	crash: read error: kernel virtual address: c0001e2d2fe4c000  type: "softirq thread_union"
	crash: cannot read softirq_ctx[930] at c0001e2d2fe48000
	crash: compressed kdump: uncompress failed: 0
	crash: read error: kernel virtual address: c0001e2d2fe44000  type: "softirq thread_union"
	crash: cannot read softirq_ctx[931] at c0001e2d2fe40000
Workaround

Disabling multi-threading in makedumpfile is the only workaround. You can use the following command to patch the kdump scripts to disable multi-threading:

sed -i 's/THREADS=\$(nproc)/THREADS=0/' /lib/dracut/modules.d/99kdumpbase/kdump.sh

The above kdump.sh script is installed by the kdump-utils package and is responsible for appending the --num-threads flag to enable multi-threading in makedumpfile.
In the above command, the string THREADS=$(nproc) is replaced with THREADS=0, which causes the script to not append the --num-threads option to makedumpfile.
This command causes the kdump service to use the makedumpfile, without multi-threading, as it used to in earlier releases. 

You must restart the kdump service for the changes to take effect by running the following command:

systemctl restart kdump.service
Fix Outlook

RHEL bug: 93694 (Fix is identified and will be included in future updates)

I/O device impacted

Not applicable. 

[{"Type":"MASTER","Line of Business":{"code":"LOB69","label":"Storage TPS"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SGMV157","label":"IBM Support for Red Hat Enterprise Linux Server"},"ARM Category":[{"code":"a8m0z000000Gnl7AAC","label":"Red Hat Enterprise Linux"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.4.0;7.5.0;7.6.0;7.7.0;7.8.0;7.9.0;8.0.0;8.1.0;8.2.0;8.3.0"}]

Document Information

Modified date:
23 June 2025

UID

ibm17236632

Page Feedback