IBM Spectrum Scale Alert : Active AFM DR Relationships goes to the Unmounted state.

Content

IBM has identified a problem in IBM Spectrum Scale versions 5.0.5.2 and 5.0.5.3, where after a successful upgrade to the mentioned versions, previously Active AFM DR relationships will move to the Unmounted state and fail to go the Active state. This state of AFM DR relationships will halt the data replication from the Production site to the DR site completely. For AFM caching modes, the relationship will remain in the Active state and replicate/cache contents as expected, but certain IBM Spectrum Scale specific Attributes may be skipped for transfer (like Extended Attributes, Sparse file information etc.) 

Users Affected:

Users who recently upgraded from older versions of IBM Spectrum Scale to versions 5.0.5.2 (or) 5.0.5.3 and find that all or some AFM DR relationships have moved to the Unmounted state and never went the Active state thereafter.  In that case, the gateway node for such Unmounted state filesets will report the following error in the mmfs.log file :

"[E] AFM: Target for fileset xxx is not a secondary-mode fileset or file system. This is required to operate in primary mode. The fileset will be disabled."

Users who recently upgraded from older versions of IBM Spectrum Scale to versions 5.0.5.2 (or) 5.0.5.3 and find that all or some AFM Caching (modes SingleWriter/IndependentWriter/ReadOly/LocalUpdates) relationships are in the Active state and working as expected, but the Gateway node of these filesets may report the following warning in the mmfs.log file :

"[I] AFM: AFM is not enabled for fileset xxx in filesystem yyy. Some features will not be supported, see documentation for enabling AFM and unsupported features."

Problem Determination:

AFM DR relationships may move to the Unmounted state and never go to the Active state thereafter. In that case, the gateway node for such Unmounted state filesets shall report the following error in the mmfs.log file :

"[E] AFM: Target for fileset xxx is not a secondary-mode fileset or file system. This is required to operate in primary mode. The fileset will be disabled."

 AFM Caching mode relationships (modes SingleWriter/IndependentWriter/ReadOly/LocalUpdates) shall remain in the Active state and work as expected, but the Gateway node of these filesets may report the following warning in the mmfs.log file :

 "[I] AFM: AFM is not enabled for fileset xxx in filesystem yyy. Some features will not be supported, see documentation for enabling AFM and unsupported features."

Recommendations:

Affected users should upgrade the Secondary/Home/DR Site to IBM Spectrum Scale version 5.0.5.2 with APAR IJ28611 efixed or 5.0.5.3 with APAR IJ28611 efixed as soon as possible.

It is recommended that users who want to upgrade to IBM Spectrum Scale version 5.0.5.3 also apply 5.0.5.3 with APAR IJ28611 efixed and then perform a startup on the DR Site.

If upon applying the efix, the problem (shown in Problem Determination section) persists (or)

Until a fix (in PTF or efix form) is applied, when either symptom (shown in Problem Determination section) is observed, the user should:

1) At the Primary/Cache site, stop replication on the fileset in question using command "mmafmctl <fs> stop -j <filesetName>"

2) At the Secondary/DR/Home site, run command "mmafmconfig disable <filesetExportPath>"

3) At the Secondary/DR/Home site, run command "mmafmconfig enable <filesetExportPath>"

4) At the Secondary/DR/Home site's NFS Server Node(s), restart the NFS Service

5) At the Primary/Cache site, start replication on the fileset in question using command "mmafmctl <fs> start -j <filesetName>"