Topic
  • 1 reply
  • Latest Post - ‏2013-01-10T20:08:03Z by vpaul
SystemAdmin
SystemAdmin
2092 Posts

Pinned topic GPFS volume cannot be mounted

‏2013-01-10T06:42:01Z |
Hi All,

We recently experienced a hardware failure on a DS3512 holding GPFS volume. The hardware has been replaced, but the GPFS volume refuses to be mounted. The relevant log from the mmfs.log is:


Removing old /var/adm/ras/mmfs.log.* files: Unloading modules from /lib/modules/2.6.32-71.el6.x86_64/extra Loading modules from /lib/modules/2.6.32-71.el6.x86_64/extra Module                  Size  Used by mmfs26               1607434  0 mmfslinux             267034  1 mmfs26 tracedev               29486  2 mmfs26,mmfslinux Wed Jan  9 21:41:41.163 2013: mmfsd initializing. 
{Version: 3.4.0.6   Built: May 27 2011 16:04:39
} ... Wed Jan  9 21:41:44.797 2013: This node (169.254.100.41 (nsd-s1-10Gb)) is now Cluster Manager 

for nsd-s1-10Gb. Wed Jan  9 21:41:44.847 2013: mmfsd ready Wed Jan  9 21:41:44 SGT 2013: mmcommon mmfsup invoked. Parameters: 169.254.100.41 169.254.100.41 all Wed Jan  9 21:41:44 SGT 2013: mounting /dev/gpfs0 Wed Jan  9 21:41:44.969 2013: Command: mount gpfs0 Wed Jan  9 21:41:45.053 2013: Node 169.254.100.41 (nsd-s1-10Gb) appointed as manager 

for gpfs0. Wed Jan  9 21:41:45.470 2013: Recovery Log I/O Failed, Unmounting file system gpfs0 Wed Jan  9 21:41:45.471 2013: Log recovery failed. Wed Jan  9 21:41:45.470 2013: File System gpfs0 unmounted by the system with 

return code 225 reason code 225 Wed Jan  9 21:41:45.471 2013: Log recovery failed. Wed Jan  9 21:41:45.470 2013: Grace period thread started Wed Jan  9 21:41:45.483 2013: Failed to open gpfs0. Wed Jan  9 21:41:45.484 2013: Log recovery failed. Wed Jan  9 21:41:45.483 2013: Command: err 225: mount gpfs0 Wed Jan  9 21:41:45.484 2013: Log recovery failed.


mmlsdisk output:

disk         driver   sector failure holds    holds                                    storage name         type       size   group metadata data  status        availability disk id pool         remarks ------------ -------- ------ ------- -------- ----- ------------- ------------ ------- ------------ --------- nsd_sdb      nsd         512    4001 yes      yes   ready         up                 1 system        desc nsd_sdc      nsd         512    4002 yes      yes   ready         up                 2 system        desc nsd_sdd      nsd         512    4001 yes      yes   ready         up                 3 system        desc nsd_sde      nsd         512    4002 yes      yes   ready         up                 4 system

Based on the log and suggestions from the internet, I ran mmfsck to check for any corruption. The command I used is:


mmfsck gpfs -v -n > mmfsck.gpfs.out 2>&1


and from the result (attached) I can see that there are several things that needed to be fixed. I want to get some opinion before I run the repair and fix the issue permanently. Please advise. Thanks.
Updated on 2013-01-10T20:08:03Z at 2013-01-10T20:08:03Z by vpaul
  • vpaul
    vpaul
    81 Posts

    Re: GPFS volume cannot be mounted

    ‏2013-01-10T20:08:03Z  
    Hello,

    Given the multiple issues mmfsck is hitting, you should go ahead and open a PMR with IBM service.

    Thanks.