Troubleshooting
Problem
Starting in mid-November 2021 multiple sites reported that DataStage parallel jobs would hang either during job init, or at end of job run.
The start of problem did not coincide with any change to operating system or DataStage software. The problem begins when site deploys an update to CrowdStrike falcon-sensor software for script-based execution monitoring.
The start of problem did not coincide with any change to operating system or DataStage software. The problem begins when site deploys an update to CrowdStrike falcon-sensor software for script-based execution monitoring.
Symptom
DataStage parallel jobs hang either at beginning of job after a few messages, or at very end of job run. Whether job hangs at start or end depends upon whether or not a pipe from prior job run remains in project_name/SC_TEMP directory, and whether or not job is called from a sequence.
For example, a newly created job hangs at end of job run (when it tries to remove pipe file created by job). If job is reset and run from sequence, the job can hang at start of job run when it tries to clean up the pipe file from the prior job run.
When a job hangs at the start of job run, the log might contain the following event messages:
In some cases, a job hanging during initialization might also display the following message before hanging:
When job hangs at end of job run, you might see the message:
However, the job log is missing the final control message and job remains in "running" state:
For example, a newly created job hangs at end of job run (when it tries to remove pipe file created by job). If job is reset and run from sequence, the job can hang at start of job run when it tries to clean up the pipe file from the prior job run.
When a job hangs at the start of job run, the log might contain the following event messages:
Message: Starting Job jobname.
Message: Environment variable settings:
In some cases, a job hanging during initialization might also display the following message before hanging:
Parallel job default NLS map UTF-8, default locale OFF
When job hangs at end of job run, you might see the message:
Parallel job reports successful completion
However, the job log is missing the final control message and job remains in "running" state:
Finished Job jobname.
Another symptom of this issue occurs in the output of the following command, which shows 2 DSD.RUN processes for each active job, instead of the usual 1. One process is the parent of the other process:
ps -ef | grep DSD.RUN
dsadm 14834 14827 0 14:06 ? 00:00:00 phantom DSD.RUN Job_name 0/1000000/0/0/0/0/0/
dsadm 14982 14834 0 14:06 ? 00:00:00 phantom DSD.RUN Job_name 0/1000000/0/0/0/0/0/
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB76","label":"Data Platform"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSZJPZ","label":"IBM InfoSphere Information Server"},"ARM Category":[{"code":"a8m50000000L0uQAAS","label":"DataStage-\u003EJob Runtime Issues"}],"ARM Case Number":"TS007502508","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"}],"Version":"11.5.0;11.7.0;11.7.1"}]
Log InLog in to view more of this document
This document has the abstract of a technical article that is available to authorized users once you have logged on. Please use Log in button above to access the full document. After log in, if you do not have the right authorization for this document, there will be instructions on what to do next.
Was this topic helpful?
Document Information
Modified date:
25 April 2025
UID
ibm16525330