Topic
  • 2 replies
  • Latest Post - ‏2019-05-22T11:03:27Z by MihaiBoncalo
MihaiBoncalo
MihaiBoncalo
2 Posts

Pinned topic Failed to read a file system descriptor. Input/output error

‏2019-05-22T10:09:20Z | descriptor filesystem gpfs4.1

Good day,

GPFS 4.1

After a team rebooted the server for patching, one filesystem is not coming up:

 

Wed May 22 04:04:33.118 2019: [I] Command: mmchmgr /dev/HA1 stage1
Wed May 22 04:04:33.531 2019: [N] Node 10.155.126.168 (stage1) appointed as manager for HA1.
Wed May 22 04:04:33.532 2019: [I] Command: successful mmchmgr /dev/HA1 iib01-dal09-staging.whirlpool.com
Wed May 22 04:04:43.191 2019: [I] Command: mount HA1
Wed May 22 04:04:43.509 2019: Failed to read a file system descriptor.
Wed May 22 04:04:43.510 2019: [E] File system manager takeover failed.
Wed May 22 04:04:43.509 2019: Input/output error
Wed May 22 04:04:43.510 2019: [X] File System HA1 unmounted by the system with return code 212 reason code 5
Wed May 22 04:04:43.509 2019: The current file system manager failed and no new manager will be appointed.
Wed May 22 04:04:43.510 2019: Failed to open HA1.
Wed May 22 04:04:43.511 2019: Input/output error
Wed May 22 04:04:43.510 2019: [W] Command: err 5: mount HA1
Wed May 22 04:04:43.511 2019: Input/output error
Wed May 22 04:04:43 CDT 2019: mmcommon preunmount invoked.  File system: HA1  Reason: SGPanic
Wed May 22 04:04:43.584 2019: [N] Node 10.155.126.168 (stage1) resigned as manager for HA1.
Wed May 22 04:04:43.585 2019: Input/output error
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

[root@stage1 ~]# mmnsddiscover -a -N all
mmnsddiscover:  Attempting to rediscover the disks.  This may take a while ...
mmnsddiscover:  Finished.
[root@stage1 ~]# mmlsnsd -M

 Disk name    NSD volume ID      Device         Node name                Remarks
---------------------------------------------------------------------------------------
 HA1          0A9B7EA85715604B   /dev/dm-1      stage2
 HA1          0A9B7EA85715604B   -              stage1 (not found) server node
 HA1          0A9B7EA85715604B   -              stage3 (not found) server node
 HA2          0A9B7EA8571560EF   /dev/dm-2      stage2 server node
 HA2          0A9B7EA8571560EF   -              stage1 (not found) server node
 HA3          0A9B7EBA5715615D   /dev/dm-3      stage2 server node
 HA3          0A9B7EBA5715615D   -              stage2 (not found) server node

[root@stage1 ~]# mmchdisk HA1 start -a
mmnsddiscover:  Attempting to rediscover the disks.  This may take a while ...
mmnsddiscover:  Finished.
stage1:  Rediscovery failed for HA1.
mmdsh: stage1 remote shell process had return code 216.
stage3:  Rediscovery failed for HA1.
mmdsh: stage3 remote shell process had return code 216.
mmnsddiscover: Command failed for one or more of the disks on one or more of the nodes.
    Examine previous error messages to determine cause.
mmchdisk: Processing continues ...
Failed to read a file system descriptor.
Input/output error
mmchdisk: Command failed. Examine previous error messages to determine cause.
[root@stage1 ~]# mmgetstate -a

 Node number  Node name        GPFS state
------------------------------------------
       1      stage1 active
       2      stage2 active
       3      stage3 active
[root@stage1 ~]# mmfsck HA1
Failed to read a file system descriptor.
Input/output error
mmfsck: Command failed. Examine previous error messages to determine cause.
[root@stage1 ~]# mmexportfs HA1 -o export.out

mmexportfs: Processing file system HA1 ...
mmexportfs: GPFS configuration data for file system HA1
may not be in agreement with the on-disk data for the file system.
Issue the command:
  mmcommon recoverfs HA1
mmexportfs: Command failed. Examine previous error messages to determine cause.
[root@stage1 ~]# mmcommon recoverfs HA1
Verifying file system configuration information ...
Failed to read a file system descriptor.
Input/output error
mmcommon: Failed to collect required file system attributes.
mmcommon: Unexpected error from reconcileSdrfsWithDaemon.  Return code: 1
mmcommon: Command failed. Examine previous error messages to determine cause.
 

IS there a way to fix this ?

 

Thank you.

 

 

  • truongv
    truongv
    103 Posts
    ACCEPTED ANSWER

    Re: Failed to read a file system descriptor. Input/output error

    ‏2019-05-22T10:59:29Z  

    The underlying disk subsystem must be fixed first.

  • truongv
    truongv
    103 Posts

    Re: Failed to read a file system descriptor. Input/output error

    ‏2019-05-22T10:59:29Z  

    The underlying disk subsystem must be fixed first.

  • MihaiBoncalo
    MihaiBoncalo
    2 Posts

    Re: Failed to read a file system descriptor. Input/output error

    ‏2019-05-22T11:03:27Z  
    • truongv
    • ‏2019-05-22T10:59:29Z

    The underlying disk subsystem must be fixed first.

    Indeed, sata link seems to be down and device mapper failed. Thank you!