IJ11334: TCPCONNECTIONS MONITOR: EXCEPTION ON WRITE FILE /GPFS/FS0/CES/C

APAR status

• Closed as program error.

Error description

• A message like "exception on write file
  /gpfs/fs0/ces/connections .... [Errno 2] No such file or
  directory" showed up in the mmfs.log file, which indicates
  a file creation problem in the sharedroot folder.
  The created connection file might contain invalid data, so that
  a NFS failover might not inform the affected clients about the
  IP address move.
  

Local fix

• na
  

Problem summary

• A message like "exception on write file
  /gpfs/fs0/ces/connections .... [Errno 2] No such file or
  directory" showed up in the mmfs.log file, which indicates
  a file creation problem in the sharedroot folder.
  The created connection file might contain invalid data, so that
  a NFS failover might not inform the affected clients about the
  IP address move.
  

Problem conclusion

• Fixed the code so that the temporary connection information
  file is created on a local filesystem, before it is copied
  to the sharedroot directory. Connection files are now only
  created when the corresponding IP address is indeed hosted
  (as shown by 'ip addr').
  
  Work Around:      None
  
  Problem trigger :
  A  CES-IP was removed from node A and moved to node B
  (failover).
  'ip addr'  showed, that this IP was indeed not hosted on
  node A any more, but now hosted on node B.
  That works as expected. However, the "ss -nt state established"
  command (and also netstat) reported that IP still on node A.
  The reason is not clear. The "rpcbind" had a process running,
  which used that IP ( that is unexpected). Development has not
  seen such a situation before. It could be OS dependent.
  Since the IP was indeed hosted on node B, both nodes tried to
  create a temp file (connection information) for the same IP
  directly in the sharedroot folder. So node A  finished the
  writing of that temp file and renamed it to its final name.
  When node B came to that point, the temp file was not there
  any more (because of the rename by node A), and the reported
  error " [Errno 2] No such file or directory"  was logged in
  mmfs.log.
  
  Symptom:  Error output/message
  
  Platforms affected:  Linux Only (CES nodes)
  
  Functional Area affected: CES
  
  Customer Impact: Medium Importance
  
  Changed Externals:None
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SPEC SCALE STD

• Fixed component ID

  5737F33AP

Applicable component levels

• R502 PSY U883600

     UP18/12/18 I 1000