Best practices for using the bulk load program

You should load multiple input files in the correct order (according to time or alphabetically), refresh your latest files, and remove files from your shared directory before they expire from the processed list file.

There are two approaches to control the order in which multiple input files from a directory are loaded. One option is to load each file individually, loading the files in the correct order. The approach might be necessary if the only difference between the file names is a time stamp in the file name. A second option is to change the names of the files to include alphabetic ordering strings. These ordering strings are then defined to the bulk load program using the processOrder.list file. The processOrder.list file does not exist, therefore you must create it manually. The bulk load program processes files that match the first ordering string first, the second ordering string second and so on. If more than one file matches the same ordering string, a processing order within that group is not ensured.

For refresh files, typically only the latest refresh file should be loaded. For refresh and delta files, the refresh file should typically be loaded first and then the delta files are loaded in the sequence in which they were generated. For just delta files, they should be loaded in the sequence in which they were generated.

A shared directory used for input files must be properly maintained. Loaded input files must be removed from the shared directory before they are expired from the processed list file. If a file remains in the directory after being expired from the processed list, it is reloaded, perhaps loading older data.

Frequent performance of database tuning during the initial DLA books loads

TADDM requires database maintenance to optimize resource usage and improve performance of SQL queries. The TADDM bulk load program (loadidml.sh file for UNIX systems or loadidml.bat for Windows systems) reads and updates database objects while moving the data from DLA books into the TADDM database. The successful maintenance requires:
  • Loading the representative data into the database that is later used for statistics calculation.
  • Performing TADDM database maintenance according to the Database maintenance topic, to gather database statistics.
Performing database maintenance frequently during the initial DLA books loads significantly improves performance and reduces the time that it takes SQL queries (SELECT, UPDATE, DELETE) to run against the database. After the initial DLA books loads, this process is not required because the database statistics are valid.
Fix Pack
9Note: z/VM DLA is not supported.
Performing database maintenance during the initial DLA books loads.
The following example refers to the z/OS DLA books.
  1. Load each type of a book from one of your smaller LPARs (BASE, TASK, DB2, IMS, CICS, ZOS, MQ, WAS).
  2. Perform database maintenance to gather the database statistics. For DB2, you must run RUNSTATS/REORG statements.
  3. Load all the books from one of your smaller LPARs (BASE, TASK, DB2, IMS, CICS, ZOS, MQ, WAS).
  4. Perform database maintenance again to update the database statistics.
  5. Load all the books from one of your largest LPARs (BASE, TASK, DB2, IMS, CICS, ZOS, MQ, WAS).
  6. Perform database maintenance again to update the database statistics.
  7. Load the rest of the books. Perform database maintenance online during the bulk load process every couple of hours to gather the latest database statistics.