HD85642: WHAT IS THE SIGNIFICANCE OF THE LARGE NUMBER OF THREADSOCKETCHE KER MESSAGES?

APAR status

• Closed as user error.

Error description

• What is the significance of the large number of
  ThreadSocketCheker messages?
  Customer is currently working on connection error
  issues with users losing their client session or it
  locking up with 'Connection Error' showing on the
  UI.
  They are receiving many ThreadSocketCheker
  messages.  The develoment responded with the
  following on this issues: " This thread is
  here to force the shutdown of the server in case of
  a disconnection of the client. So if you have the
  lines "ThreadSocketCheker is trying to force the
  server to stop" that will mean that there is a
  disconnection, other way this is not the case"
  The customer reports that they have 179 sessions
  that had the message in their vpmsrv log in the
  past two days.
  Please advise additional messages or traces that
  the customer can look at or activate to identify if
  this is a problem causing their connection issues.
  Customer info --------
  We are seeing a number of sessions with the
  indicated "ThreadSocketCheker is trying to force
  the server to stop"
  Below is part of one of the sessions showing the
  message stated above.
  08:08:15:595 to797c 1491532:
  ThreadSocketCheker has detected a
  selecrc=<1>
  08:08:15:595 to797c 1491532:
  ThreadSocketCheker
  FD_ISSET(_Fd,&sel_read)=<1>
  08:08:15:595 to797c 1491532:
  ThreadSocketCheker nboctet in
  ioctl=<0>
  cout
  BENBBESendBBDICommand::CustomAppExecute
  08:11:20:272 to797c 1491532:
  ThreadSocketCheker has detected a
  selecrc=<1>
  08:11:20:272 to797c 1491532:
  ThreadSocketCheker
  FD_ISSET(_Fd,&sel_read)=<1>
  08:11:20:272 to797c 1491532:
  ThreadSocketCheker nboctet in
  ioctl=<152>
  08:51:42:115 to797c 1491532:
  ThreadSocketCheker has detected a
  selecrc=<1>
  08:51:42:115 to797c 1491532:
  ThreadSocketCheker
  FD_ISSET(_Fd,&sel_read)=<1>
  08:51:42:115 to797c 1491532:
  ThreadSocketCheker nboctet in
  ioctl=<157>
  08:51:49:270 to797c 1491532:
  CATFdManagement::addFdBot: 43
  08:51:49:270 to797c 1491532: CB Orbix Open
  fd:43 8 with cb
  [000010f14a2e64c5:New IIOP Connection
  (130.247.45.122:52567) ]
  08:51:49:270 to797c 1491532:
  CATAdminServerIDLMSLightImpl stop
  08:51:49:270 to797c 1491532: XCO modif loop
  after pxc0loop:18049599
  08:51:49:270 to797c 1491532: Server ended in
  time
  out:ServerTimeOut|000010f14a2e64c5
  08:51:49:270 to797c 1491532: IN:
  CATDataAvailableSourceORBImplementation::fireD
  ataAvailable("ServerTimeOut|000010f14a2e64c5")
  08:51:49:270 to797c 1491532: number of
  Listeners: 0
  08:51:49:270 to797c 1491532: OUT:
  CATDataAvailableSourceORBImplementation::fireD
  ataAvailable("ServerTimeOut|000010f14a2e64c5")
  08:51:54:2 to797c 1491532: ThreadSocketCheker
  has detected a
  selecrc=<1>
  08:51:54:2 to797c 1491532: ThreadSocketCheker
  FD_ISSET(_Fd,&sel_read)=<1>
  08:51:54:2 to797c 1491532: ThreadSocketCheker
  nboctet in
  ioctl=<0>
  08:51:54:2 to797c 1491532: ThreadSocketCheker
  has detected a socket
  close (ioctlrc=<-1>)
  08:51:54:2 to797c 1491532: ThreadSocketCheker
  let 2 minutes for the
  server to stop (case of normal stop)
  08:51:59:270 to797c 1491532: We are out of
  CORBA loop
  08:51:59:270 to797c 1491532: Implementation
  deactivated
  [ 08:51:59.285
  000010f14a2e64c5@prodeno21g.ca.boeing.com
  <mailto:000010f14a2e64c5@prodeno21g.ca.boeing.
  com> Outgoing
  request `sendCommitBuffer' to
  prodvltg.ca.boeing.com@ENOVIAVaultServer
  <mailto:prodvltg.ca.boeing.com@ENOVIAVaultServ
  er> ] prodvltg.ca.boeing.com@ENOVIAVaultServer
  <mailto:prodvltg.ca.boeing.com@ENOVIAVaultServ
  er> ]
  [ 08:51:59.294
  000010f14a2e64c5@prodeno21g.ca.boeing.com
  <mailto:000010f14a2e64c5@prodeno21g.ca.boeing.
  com> Incoming reply
  to `sendCommitBuffer' from
  ENOVIAVaultServer@prodvltg.ca.boeing.com
  <mailto:ENOVIAVaultServer@prodvltg.ca.boeing.co
  m> ] ENOVIAVaultServer@prodvltg.ca.boeing.com
  <mailto:ENOVIAVaultServer@prodvltg.ca.boeing.co
  m> ]
  [ 08:52:06.976
  000010f14a2e64c5@prodeno21g.ca.boeing.com
  <mailto:000010f14a2e64c5@prodeno21g.ca.boeing.
  com> Outgoing
  request `sendCommitBuffer' to
  prodvltg.ca.boeing.com@ENOVIAVaultServer
  <mailto:prodvltg.ca.boeing.com@ENOVIAVaultServ
  er> ] prodvltg.ca.boeing.com@ENOVIAVaultServer
  <mailto:prodvltg.ca.boeing.com@ENOVIAVaultServ
  er> ]
  [ 08:52:07.029
  000010f14a2e64c5@prodeno21g.ca.boeing.com
  <mailto:000010f14a2e64c5@prodeno21g.ca.boeing.
  com> Incoming reply
  to `sendCommitBuffer' from
  ENOVIAVaultServer@prodvltg.ca.boeing.com
  <mailto:ENOVIAVaultServer@prodvltg.ca.boeing.co
  m> ] ENOVIAVaultServer@prodvltg.ca.boeing.com
  <mailto:ENOVIAVaultServer@prodvltg.ca.boeing.co
  m> ]
  08:53:54:3 to797c 1491532: ThreadSocketCheker
  is trying to force the server to stop.
  .
  

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

• Incident Diagnosis
  This thread is here to kill the server in case of
  client disconnection (client crash or user kill the
  client). The goal is to prevent a server to compute
  for many hours without being able to reply to the
  client and also doing a useless rollback for many
  hours too.
  Of course this thread will also kill the server in case
  of any disconnection between client and server
  (firewall, cache, antivirus ...).
  If customer is facing lines like the one just under in
  traces there is of course many chance to be in one
  of the above describe case:
  08:53:54:3 to797c 1491532: ThreadSocketCheker
  is trying to force the server to stop.
  PS: client can be on server side if a server
  connects to another one (like enovia with vault)
  Reason
  Search on client side the reason of the
  disconnection, if the server traces shows (and only
  in that case):
  ThreadSocketCheker is trying to force the server
  to stop.
  Additional Closure Information:
  .
  

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

Applicable component levels