Problems and solutions for preventing WebSphere eXtreme Scale container OutOfMemory error during startup

Edit online
Test test When multiple WebSphere eXtreme Scale container members are started concurrently, some containers may become overloaded with data, causing one or more container members to run out of memory and terminate ungracefully. If this occurs, AS4 Microservices may become unavailable.

Symptoms

When multiple WebSphere eXtreme Scale containers are started concurrently, some containers may become overloaded with data, causing OutOfMemory errors and possible rendering B2B Advanced Communication unavailable.

To prevent this from occurring, follow the below procedure during WebSphere eXtreme Scale container startup when either of the following conditions were true when the B2BAC cluster was last in use:
  • Memory utilization in the eXtreme Scale grid was high (60% or more).
  • AS4 Microservices was in an interval in which either the low or high memory watermark thresholds had been reached. See Configuring the data grid cache memory watermarks for more information about memory watermarks.

Resolving the problem

To gracefully start a AS4 Microservices cluster in which the WebSphere eXtreme Scale container members have previously had high memory utilization:

  1. Start each container member.
  2. On each AS4 Microservices node using Linux with a WebSphere eXtreme Scale container member installed, open the this file in a text editor on each of the five grids: <b2bacinstall>/Members/WXSContainer /usr/servers/WXSContainer/grids/<gridname>/objectGridDeployment.xml. The <b2bacinstall> directory is the AS4 Microservices install directory. The <gridname> is one of the following:
    • Base
    • Comms
    • Identity
    • Infrastructure
    • MessageFabirc
    Important: The abovementioned directory is correct for Linux. When running B2B Advanced Communications on Windows, open the this file in a text editor on each of the five grids: <b2bacinstall>\Members\WXSContainer \usr\servers\WXSContainer\<gridname>\objectGridDeployment.xml. The <b2bacinstall> directory is the AS4 Microservices install directory. The <gridname> is one of the following:
    • Base
    • Comms
    • Identity
    • Infrastructure
    • MessageFabirc
  3. On each AS4 Microservices node with a WebSphere eXtreme Scale container member installed, modify the objectGridDeployment.xml as follows. Examine each mapSet XML element and add a new numInitialContainers="<N>" attribute, where <N> is the number of WebSphere eXtreme Scale container members in your cluster.
    For example, suppose you have a cluster containing three WebSphere eXtreme Scale container members, and you need to modify the following mapSet element: 
    <mapSet name="SFSNonTrxDataMapSet"
     numberOfPartitions="5"
     minSyncReplicas="0"
     developmentMode="false">
     <map ref=<SFSDestinationInfoMap" />
</mapSet>
    You need to modify the element as follows:
    <mapSet name="SFSNonTrxDataMapSet"
     numInitialContainers="3" <!--NEW Attribute -->
     numberOfPartitions="5"
     minSyncReplicas="0"
     developmentMode="false">
     <map ref=<SFSDestinationInfoMap" />
</mapSet>
    Remember: Modify all mapSet elements on the objectGridDeployment.xml file for each WebSphere eXtreme Scale container
  4. Start each container member.
  5. Start each operational member.
    Note: As usual, each operational member will state that it is waiting for the grid while the preload completes.
  6. Start each informational member.
    Note: As usual, each informational member will state that it is waiting for the grid while the preload completes.