IBM Support

JR35518: HEAP ALLOCATION FAILURE IN THE MQ CONNECTOR STAGE WHEN THE ROW BUFFER COUNT MESSAGE OPTION IS USED.

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • The MQ Connector stage can consume multiple input rows from a
    link, buffer them in memory, and then create a single output
    message with a payload of all the rows consumed. This feature is
    enabled by setting the row buffer count message option to -1.
    The problem is that the memory usage is extremely high when this
    feature is used.  Even small volumes of input data result in
    massive memory usage and ulimately heap allocation failures,
    similar to the following:
    
    Fatal Error: Throwing exception: APT_BadAlloc: Heap allocation
    failed.
    

Local fix

  • The MQ Plugin stage (can be used in server or parallel jobs) is
    capable of similar (but more limited) functionality and does
    not have this problem.
    

Problem summary

  • ****************************************************************
    USERS AFFECTED:
    MQ connector users with row buffer count set
    ****************************************************************
    PROBLEM DESCRIPTION:
    Job failed when row buffer count property set for MQ connector.
    
    There was a program error on the code path handling row buffer
    count property to concatenate multiple input data to create a
    message.
    There was a problem in the memory allocation logic, and memory
    leak occurred.
    When MQ connector runs handling the row buffer count, the osh
    process grows very rapidly up to 1-2 GB and then reaches limit
    and the job fails.
    
    ****************************************************************
    RECOMMENDATION:
    Apply the patch.
    ****************************************************************
    

Problem conclusion

  • The code has been fixed to correctly free up the buffer. The
    allocation logic is enhanced as well to use the memory
    effectively.
    

Temporary fix

  • Avoid using Row Buffer Count property and concatenate the data
    outside MQ connector.
    

Comments

APAR Information

  • APAR number

    JR35518

  • Reported component name

    WIS DATASTAGE

  • Reported component ID

    5724Q36DS

  • Reported release

    801

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2010-01-29

  • Closed date

    2010-03-09

  • Last modified date

    2010-03-09

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Modules/Macros

  • SERVER
    

Fix information

  • Fixed component name

    WIS DATASTAGE

  • Fixed component ID

    5724Q36DS

Applicable component levels

  • R801 PSN

       UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSVSEF","label":"InfoSphere DataStage"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.0.1","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
12 October 2021