PI96954: LIBERTY ON Z/OS MEMORY LEAK IN 64BIT PRIVATE DUE TO NATIVE DIRECT BYTEBUFFER SUPPORT

Fixes are available

APAR status

Closed as program error.

Error description

At 17.0.0.2 and beyond, work done for the split to
commercial
and open source versions of liberty accidentally enabled
native direct ByteBuffer support. This support is enabled
regardless of whether the z/OS asyncio support in liberty is
in use or not.
On common paths for http response flush or inbound request
handling the memory obtained this way is not freed.
This leads to a leak in 64 bit private.
In the customer case this was for both LE HEAPPOOL64
cellpool
11 and 5, although this might vary depending on the request
payload / bodies.
This APAR will disable this support while additional testing
is done. It will also resolve a small cellpool 5 leak
associated
with AsyncIO CompletionKey and ResultHandler classes.

Local fix

Problem summary

****************************************************************
* USERS AFFECTED:  All users of IBM WebSphere Application      *
*                  Server Liberty for z/OS                     *
****************************************************************
* PROBLEM DESCRIPTION: Liberty on z/OS memory leak in 64-bit   *
*                      private due to custom DirectByteBuffer  *
*                      allocation support                      *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
The WebSphere Liberty for z/OS Async I/O feature includes a
custom library for allocating DirectByteBuffers.  This custom
support does not include automatic memory management and
requires the caller to explicitly delete the buffer when it is
no longer in use.  This support was inadvertently enabled for
use beyond the Async I/O feature, causing a leak of the byte
buffers in areas of code that rely on automatic memory
management.

The WebSphere Liberty code has defined pool sizes that it uses
for allocation of DirectByteBuffers, therefore the size of the
leaked areas relate directly to these pool sizes and can vary.
For example, in some cases the leaked storage will contain HTTP
requests and responses and the size of those requests and
responses will dictate which pool size is used for the native
allocation request.

An LE Heappool Trace (HPT) of this issue will show a pattern of
allocations from native routine
"com/ibm/ws/zos/asyncio/ZOSAsyncIOSupport::allocateDirectByteBuf
fer(J)Ljava/nio/ByteBuffer;" in libzNativeServices.so on
executor threads named Default Executor-thread-<int>, with no
corresponding releaseDirectByteBuffer() calls.

Also, fixed within this APAR is a small leak in the
CompletionKey class.   This class uses DirectByteBuffers for
managing its I/O requests.  The size of these allocation
requests fit into the smallest buffer pool of 1024 in size.

Problem conclusion

Code has been modified to disable the custom native routines for
allocateDirectByteBuffer and releaseDirectByteBuffer.  The
WebSphere Liberty code now uses the JDK
java.nio.ByteBuffer.allocateDirect method for obtaining
DirectByteBuffers

Code has also been modified in the Async I/O channel to properly
re-use and free the related DirectByteBuffer allocations.  This
issue is fixed by Open Liberty GitHub issue 3160.  For more
information, see https://github.com/OpenLiberty/open-
liberty/issues/3160.


The fix for this APAR is currently targeted for inclusion in fix
pack 18.0.0.2.  Please refer to the Recommended Updates page for
delivery information:
http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980

Temporary fix

Comments

APAR Information

APAR number
PI96954
Reported component name
LIBERTY PROF -
Reported component ID
5655W6514
Reported release
CD0
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2018-04-19
Closed date
2018-05-02
Last modified date
2018-05-02

APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name
LIBERTY PROF -
Fixed component ID
5655W6514

Applicable component levels

RCD0 PSY
UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS7K4U","label":"WebSphere Application Server for z\/OS"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"CD0","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
04 May 2022

Tips

PI96954: LIBERTY ON Z/OS MEMORY LEAK IN 64BIT PRIVATE DUE TO NATIVE DIRECT BYTEBUFFER SUPPORT

Fixes are available

Subscribe

APAR status

Closed as program error.

Error description

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

APAR Information

APAR number

Reported component name

Reported component ID

Reported release

Status

PE

HIPER

Special Attention

Submitted date

Closed date

Last modified date

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name

Fixed component ID

Applicable component levels

RCD0 PSY

Document Information

Share your feedback

Need support?