IJ29025: RDMA CONNECTION CAN NOT BE ESTABLISHED IF SETTING THE EXTERNAL IPV6 ADDRESS ON AN ROCE CLUSTER

APAR status

• Closed as program error.

Error description

• Given a Spectrum Scale cluster with 'verbsRdmaCm'
  set to 'enable' and configured to use RDMA via RoCE,
  individual nodes will fail to establish a RDMA connection
  to other nodes when the IP addresses configured on
  the RDMA adapters include a non-link local IPv6 address.
  

Local fix

• Remove all non-link local IPv6 addresses from all RDMA
  adapters which are configured to be used by
  Spectrum Scale.
  

Problem summary

• Given a Spectrum Scale cluster with 'verbsRdmaCm'
  set to 'enable' and configured to use RDMA via RoCE,
  individual nodes will fail to establish a RDMA connection
  to other nodes when the IP addresses configured on
  the RDMA adapters include a non-link local IPv6 address.
  

Problem conclusion

• Benefits of the solution:
  
  The RDMA connection now can be established
  
  Work Around:
  
  Remove all non-link local IPv6 addresses from all RDMA
  adapters which are configured to be used by
  Spectrum Scale.
  
  Problem trigger:
  
  A Spectrum Scale cluster with 'verbsRdmaCm' set to 'enable'
  configured to use RDMA via RoCE and RDMA adapters
  configured with non-link local IPv6 addresses.
  
  Symptom:
  
  Performance Impact/Degradation
  
  ALL Linux OS environments
  
  Functional Area affected:
  
  RDMA
  
  Customer Impact:
  
  High
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

  IJ28631

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SPEC SCALE STD

• Fixed component ID

  5737F33AP

Applicable component levels