IBM Support

PI96954: LIBERTY ON Z/OS MEMORY LEAK IN 64BIT PRIVATE DUE TO NATIVE DIRECT BYTEBUFFER SUPPORT

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • At 17.0.0.2 and beyond, work done for the split to
    commercial
    and open source versions of liberty accidentally enabled
    native direct ByteBuffer support. This support is enabled
    regardless of whether the z/OS asyncio support in liberty is
    in use or not.
    On common paths for http response flush or inbound request
    handling the memory obtained this way is not freed.
    This leads to a leak in 64 bit private.
    In the customer case this was for both LE HEAPPOOL64
    cellpool
    11 and 5, although this might vary depending on the request
    payload / bodies.
    This APAR will disable this support while additional testing
    is done. It will also resolve a small cellpool 5 leak
    associated
    with AsyncIO CompletionKey and ResultHandler classes.
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  All users of IBM WebSphere Application      *
    *                  Server Liberty for z/OS                     *
    ****************************************************************
    * PROBLEM DESCRIPTION: Liberty on z/OS memory leak in 64-bit   *
    *                      private due to custom DirectByteBuffer  *
    *                      allocation support                      *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    The WebSphere Liberty for z/OS Async I/O feature includes a
    custom library for allocating DirectByteBuffers.  This custom
    support does not include automatic memory management and
    requires the caller to explicitly delete the buffer when it is
    no longer in use.  This support was inadvertently enabled for
    use beyond the Async I/O feature, causing a leak of the byte
    buffers in areas of code that rely on automatic memory
    management.
    
    The WebSphere Liberty code has defined pool sizes that it uses
    for allocation of DirectByteBuffers, therefore the size of the
    leaked areas relate directly to these pool sizes and can vary.
    For example, in some cases the leaked storage will contain HTTP
    requests and responses and the size of those requests and
    responses will dictate which pool size is used for the native
    allocation request.
    
    An LE Heappool Trace (HPT) of this issue will show a pattern of
    allocations from native routine
    "com/ibm/ws/zos/asyncio/ZOSAsyncIOSupport::allocateDirectByteBuf
    fer(J)Ljava/nio/ByteBuffer;" in libzNativeServices.so on
    executor threads named Default Executor-thread-<int>, with no
    corresponding releaseDirectByteBuffer() calls.
    
    Also, fixed within this APAR is a small leak in the
    CompletionKey class.   This class uses DirectByteBuffers for
    managing its I/O requests.  The size of these allocation
    requests fit into the smallest buffer pool of 1024 in size.
    

Problem conclusion

  • Code has been modified to disable the custom native routines for
    allocateDirectByteBuffer and releaseDirectByteBuffer.  The
    WebSphere Liberty code now uses the JDK
    java.nio.ByteBuffer.allocateDirect method for obtaining
    DirectByteBuffers
    
    Code has also been modified in the Async I/O channel to properly
    re-use and free the related DirectByteBuffer allocations.  This
    issue is fixed by Open Liberty GitHub issue 3160.  For more
    information, see https://github.com/OpenLiberty/open-
    liberty/issues/3160.
    
    
    The fix for this APAR is currently targeted for inclusion in fix
    pack 18.0.0.2.  Please refer to the Recommended Updates page for
    delivery information:
    http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
    

Temporary fix

Comments

APAR Information

  • APAR number

    PI96954

  • Reported component name

    LIBERTY PROF -

  • Reported component ID

    5655W6514

  • Reported release

    CD0

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2018-04-19

  • Closed date

    2018-05-02

  • Last modified date

    2018-05-02

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    LIBERTY PROF -

  • Fixed component ID

    5655W6514

Applicable component levels

  • RCD0 PSY

       UP

[{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SG19M","label":"APARs - z\/OS environment"},"Platform":[{"code":"PF054","label":"z Systems"}],"Version":"CD0","Line of Business":{"code":"","label":""}}]

Document Information

Modified date:
17 June 2020