JR51052: XML STAGE'S DISK-BASED OPERATIONS FOR HJOIN, SORT AND REGROUP STEPS HAVE VERY LARGE MEMORY FOOT PRINT.

APAR status

• Closed as program error.

Error description

• XML Stage's disk-based operations for HJoin, Sort and Regroup
  steps have very large memory foot print.
  
  The solution is be to fix the the disk-based implementation,
  reducing it's memory consumption.
  

Local fix

• 1. Sort the data records first in the DataStage and then use the
   optimized regroup operation. In this case, if the Regroup step
  is not the first step in the stage the developer should
  guarantee that no re-ordering of the data occurs prior to the
  Regroup that would cause it to become unsorted.
  
  2. Sort the data records first in the DataStage using the
  DataStage operator instead of using the Hierarchical Data stage
  sort step.
  
  3. Use the Wave Generator in DataStage to divide the data into
  waves and run the job for each wave of data. This approach only
  works for Parallel Engine jobs, not for DataStage Server jobs.
  
  4. Break a job run with a large data set into multiple job runs.
  Each run processes a subset of the large data set.
  

Problem summary

• XML STAGE'S DISK-BASED OPERATIONS FOR HJOIN, SORT AND REGROUP
  STEPS HAS VERY LARGE MEMORY FOOT PRINT
  

Problem conclusion

• Increased the default thread as 4
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  INFO SRVR PLATF

• Fixed component ID

  5724Q3612

Applicable component levels

• R850 PSY

     UP