PI22949: Rapid recycling of a jvm results in a failed server restart indi cating "java.io.IOException: The P2P layer was unable to bind to

Fixes are available

APAR status

Closed as program error.

Error description

WebSphere Virutal Enterprise V7.

Applications connect to a remote system which is shutdown
without closing the sockets.  The stale sockets result in hung
threads, and a WXD policy causes an automatic restart after the
threads are hung for a certain amount of time.

6/1/14 1:22:28:650 EDT] 00000003 TimeoutReques W   WXDH3012W:
The request timeout limit specified by policy
Default_Excessive_Request_Timeout was exceeded by server
SERVER_NAME on node NODE_NAME. The limit is 5.00 %
and the current timeout fraction is 100.00 %.  A total of 2
threads may be hung.

[6/1/14 1:22:28:650 EDT] 00000003 TimeoutReques W   WXDH3012W:
The request timeout limit specified by policy
HungThreadTimeOuts was exceeded by server SERVER_NAME on node
NODE_NAME. The limit is 80.00 % and the current timeout
fraction is 100.00 %.  A total of 2 threads may be hung.

[6/1/14 1:22:31:958 EDT] 00000260 RestartTask   I   WXDH1007I:
Server SERVER_NAME useForceToManualMode=false
clusterName=CLUSTER_NAME being restarted automatically.


Within 10 seconds the local system attempts to restart the JVM.
 When the JVM tries to start it encounters UDP port errors:

[6/1/14 1:31:50:571 EDT] 00000000 P2PBase       E   ODCF8005E:
There was an unexpected exception while initializing the SON
overlay; the exception is java.io.IOException: The P2P layer
was unable to bind to a UDP port.

Because of the UDP port error, the JVM fails to start.
However, a later restart attempt is succesful.

Local fix

If the WXD policy is present, adjust it so that the hung
threads do not result in an automatic restart of the JVM.

Problem summary

****************************************************************
* USERS AFFECTED:  All users of IBM WebSphere Virtual          *
*                  Enterprise.                                 *
****************************************************************
* PROBLEM DESCRIPTION: Recycling of a jvm results in a failed  *
*                      server restart indicating "IOException: *
*                      The P2P layer was unable to bind to a   *
*                      UDP port".                              *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
Rapid recycling of a server may result in a failed restart if
the port associated with the server's
OVERLAY_UDP_LISTENER_ADDRESS is not freed by the operating
system prior to the peer attempting to rebind to it during
server start-up.
Both FFDC and SystemOut.log will contain a message similar to
the one below; output during the failed start-up.
java.io.IOException: The P2P layer was unable to bind to a UDP
port.
at com.ibm.son.mesh.RawUDPImpl.<init>(RawUDPImpl.java:89)
at com.ibm.son.mesh.UDPFactory.getUDP(UDPFactory.java:31)
at com.ibm.son.mesh.Peer.<init>(Peer.java:470)
at com.ibm.son.mesh.Peer.<init>(Peer.java:413)
at
com.ibm.ws.p2p.P2POverlayHelper.getPeer(P2POverlayHelper.java:15
7)
at com.ibm.ws.p2p.P2PBase.getPeer(P2PBase.java:31)
at
com.ibm.ws.bbson.BBFactoryImpl.<init>(BBFactoryImpl.java:632)
at java.lang.J9VMInternals.newInstanceImpl(Native Method)
at java.lang.Class.newInstance(Class.java:1572)

Problem conclusion

The Peer-to-peer Service Overlay Network (P2P/SON) has been
altered to allow the operating system time to free the defined
OVERLAY_UDP_LISTENER_ADDRESS port rather then immediately
failing jvm start-up when the first binding failure occurs.

This fix will be included in the next available fix pack for IBM
WebSphere Virtual Enterprise.

Temporary fix

Comments

APAR Information

APAR number
PI22949
Reported component name
WAS VIRTUAL ENT
Reported component ID
5725C9203
Reported release
700
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2014-07-29
Closed date
2014-10-20
Last modified date
2014-10-20

APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name
WAS VIRTUAL ENT
Fixed component ID
5725C9203

Applicable component levels

R700 PSY
UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSUP64","label":"WebSphere Virtual Enterprise"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.0","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
28 April 2022

Tips

PI22949: Rapid recycling of a jvm results in a failed server restart indi cating "java.io.IOException: The P2P layer was unable to bind to

Fixes are available

Subscribe

APAR status

Closed as program error.

Error description

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

APAR Information

APAR number

Reported component name

Reported component ID

Reported release

Status

PE

HIPER

Special Attention

Submitted date

Closed date

Last modified date

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name

Fixed component ID

Applicable component levels

R700 PSY

Document Information

Share your feedback

Need support?