Temporary lookuptable files that use up disk space

When a InfoSphere® DataStage® job with a lookup stage fails, files with the lookuptable prefix are left in the resource directories.

Symptoms

The resource directories and common space contain files with names that are similar to the following file name: lookuptable.20091210.513biba

Causes

When a job fails, it leaves the temporary files for postmortem review in the resource directories. Temporary files are left in scratch, but lookup files are created in the resource directories. Lookup file sets are not removed. A lookup file set is similar to the following file set: /opt/IBM/InformationServer/Server/Datasets/export.dsadm.abcdefg.P000000_F0000 A lookup file has a structure that is similar to the following file: /opt/IBM/InformationServer/Server/Datasets/lookuptable.20091210.513biba

Diagnosing the problem

Look for files with file names similar to the following structure: lookuptable.yyyymmdd.nnnnnnn You can find these files on the disk when no jobs are running.

Resolving the problem

When no jobs are running, remove all of the files that have the lookuptable prefix on the file name. These files get re-created with every new run of the job and are never reused. If the job runs successfully, then only the lookuptable file that is created during that job run is removed. Create a maintenance script that cleans up the lookuptable files regularly when jobs are not running.