IBM Support

The Cron Environment and Cron Job Failures

Question & Answer


Question

Why does my job run from the command line but not from cron?

Answer

Introduction
What is the cron environment?
What is the current directory for a cron job?
How to extend the environment for a cron job
How to debug an environment problem in a cron job
Conclusion
 

Introduction

Cron is a process that reads crontab files and runs programs and scripts at the times that are specified in the crontab files. The programs and scripts that are executed by cron become child processes of cron, and thus inherit cron's process environment. This environment consists of environment variables, a working directory, umask, resource limits, etc. Sometimes a job run by cron fails, but runs successfully when run from the command line. This problem is often caused by the job's environment.

What is the cron environment?

Since the cron daemon is a system program that is started by one of the system startup scripts, its environment will not be the same as the environment inherited from the shell when a job is run from the command line. Cron inherits its environment from its parent, which is the init process. But during cron initialization, cron can modify or extend the environment before it runs cron jobs. The default environment for a cron job consists of /etc/environment, and the default shell environment variables such as $PATH, $HOME and $PWD. Information in login files (for example, /etc/profile, ~/.profile and ~/.kshrc, so it is unlikely $PATH contains login shell directories. If the cron job needs any information in these shell initialization files, these scripts must be sourced at the top of the cron job script.

System limits, also known as ulimits are essential to the cron environment. The ulimits for a cron job are inherited from the root account, and will likely not be the same as the ulimits for a regular user account.

Note: The at command is often used to test cron jobs. Because the at command is executed from the command line, unlike cron, it will automatically inherit the current environment of the shell.

What is the current directory for a cron job?

The current directory, which can be accessed using the shell environment variable $PWD, is set by cron to the home directory of the user who owns the crontab file. In other words, the current directory will not necessarily be the directory containing the shell script that is being executed by cron. For this reason relative paths should not be used within a script that is to be executed by cron, unless the current directory is explicitly set at the top of the script with the cd command.

How to extend the environment for a cron job

If a cron job uses aliases, environment variables, or functions that are defined in shell initialization scripts such as ~/.profile and ~/.kshrc, these scripts can be sourced at the top of the cron job script. Any additional information needed by a cron job can also be added directly at the top of the cron job script. For example, system ulimits might need to be explictly set in a cron job script.

The following example assumes a home directory has a .profile and a .kshrc file that need to be sourced in order to provide the proper environment for the cron job.

#!/bin/ksh
# The line above should contain whatever shell is required by the script below. By default,
# cron will use whatever the login shell is for the user that owns the crontab file.

# Duplicate the environment that is inherited when the script runs from the command line.
cd /home/$USER      # cron does this for us - this is here just to be safe
. /etc/profile

if [ -a .profile ]; then
   . ./.profile
fi

if [ -a .kshrc ]; then
   . ./.kshrc
fi

# Add any other required environment related information, such as system ulimits.
ulimit -d unlimited
ulimit -f 3000

# You can now change the current directory if desired.
cd /my/current/working/directory
...

How to debug an environment problem in a cron job

If a cron job script still fails after extending the cron job environment, then the script can be debugged by creating an environment variable list file from within the cron job script using the env command. This list can then be compared to a list file created when the job is executed successfully from the command line. Here is an example.

#!/bin/ksh
# The line above should contain whatever shell is required by the script below. By default,
# cron will use whatever the login shell is for the user that owns the crontab file.

# Duplicate the environment that is inherited when the script runs from the command line.
cd /home/$USER    # cron does this for us - this is here just to be safe
. /etc/profile

if [ -a .profile ]; then
   . ./.profile
fi

if [ -a .kshrc ]; then
   . ./.kshrc
fi

# Add any other required environment related information, such as system ulimits.
ulimit -d unlimited
ulimit -f 3000

# You can now change the current directory if you need to.
cd /my/current/working/directory

# Generate an environment list file.
usr/bin/env | sort > /tmp/cronEnv.out
...


Now create an environment list from the command line by running the env command under the account where the script executes successfully.

$ env | sort > /tmp/cmdLineEnv.out

Next compare the two environment list files to determine which variables need to be explicitly set in the script that is to be executed by cron. Variables that exist in the cmdLineEnv.out file but not in the cronEnv.out file should be added to the script. If a variable exists in both files but the values are different, use the value that is in the cmdLineEnv.out file.

# diff -b /tmp/cmdLineEnv.out /tmp/cronEnv.out

Finally, add export lines in the script to explicitly set whatever environment variables are required to completely duplicate the environment from the command line.

#!/bin/ksh

...

# Customize the environment still further by explicitly setting variables.
export myvar1=value
export myvar2=value
export myvar3=value
...

Conclusion

Cron job failures can often be traced to problems in the environment. Cron jobs are often tested in a particular user account with the at command, or by running them directly from the command line. In both cases the jobs inherit the shell environment, but this environment will not automatically be available to a job that is run from cron. This problem is easily resolved by modifying the cron job script to manually source shell startup scripts such as /etc/profile, ~/.profile, and ~/.kshrc, and by adding any additional environment variables, shell functions, or system resource limits that are required by the job.
 
Cron jobs will fail  if the max limit of jobs is exceeded. Read about the max queue definitions in the Cron daemon not running jobs technote.
SUPPORT

If you require more assistance, use the following step-by-step instructions to contact IBM to open a case for software with an active and valid support contract.  

1. Document (or collect screen captures of) all symptoms, errors, and messages related to your issue.

2. Capture any logs or data relevant to the situation.

3. Contact IBM to open a case:

   -For electronic support, see the IBM Support Community:
     https://www.ibm.com/mysupport
   -If you require telephone support, see the web page:
      https://www.ibm.com/planetwide/

4. Provide a clear, concise description of the issue.

 - See: Working with IBM AIX Support: Describing the problem.

5. Collect a system snap with cron data.

# snap -r (removes previous snap data)
# snap -aZ (excludes system dump)
# tar -cvf /tmp/ibmsupt/testcase/cron.tar -C /var adm/cron spool/cron -C /etc cronlog.conf
# ls -al /usr/sbin/cron /usr/bin/crontab /usr/bin/at > /tmp/ibmsupt/testcase/cronfiles.ls
6. Prepare the snap for upload.
# snap -c (Compress snap)
# mv /tmp/ibmsupt/snap.pax.Z /tmp/ibmsupt/yourcase#.snap.pax.Z (Rename file to associate with your case)
7. Upload all of the details and data for your case.
RECOMMENDED:
 
a) Attach to your case
 
b) Upload to the Enhanced Customer Data Repository(ECuRep)
 
ADDITIONAL OPTIONS:
 
c) Upload by Secure File Transfer
        https://testcase.software.ibm.com/
        Login with your IBMID
        Navigate to toibm/aix
        Click on File upload: "Choose File"
        Select the compressed file
        Select "Upload File (Binary)"
      
d) Blue Diamond customer data uploads (US only)

[{"Product":{"code":"SWG10","label":"AIX"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":"Support information","Platform":[{"code":"PF002","label":"AIX"}],"Version":"5.2;5.3;6.1;7.1","Edition":"","Line of Business":{"code":"LOB08","label":"Cognitive Systems"}}]

Document Information

Modified date:
20 March 2021

UID

isg3T1011623