Recovery of "Critical" status and "Write Fenced" tapes in the inventory

When a tape or tape drive in the IBM Spectrum Archive™ Enterprise Edition inventory encounters a problem, the status of the tapes that are involved can change to Critical status. You can use the ltfsee recover command to help you handle these tapes and recover data from them.

With the ltfsee recover command, migrated files can be recalled from the tape to Spectrum Scale. Other files can be copied to a specified location on Spectrum Scale or local disk.

Note: Check any Spectrum Scale policies that are used for migration. Make any necessary updates to them to remove the tape that has a status of Critical from use.

As part of the recovery operation, the tape that has a status of Critical is removed from the Spectrum Archive Enterprise Edition inventory. Then, the tape drive is made available again. If removal of the tape fails, you must move the tape from the drive by using the appropriate interface for your tape library. In this case, refer to your tape library documentation for details about operating the tape library.

For more information, see the ltfsee recover command.

Typical user scenario for recovering a tape with a "Critical" or "Write Fenced" status

The following scenario is a typical example of encountering and recovering a tape in the Spectrum Archive EE inventory that has a status of "Critical" or "Write Fenced".
  1. An error is encountered for migration, recall, reclaim, import, or export operations, and an SNMP trap notification is received.
  2. A look into ltfsee.log and ltfs.log shows that the error came from writing data.
  3. The ltfsee info tapes command is run, which shows that a tape has a status of "Critical" or "Write Fenced".
    [root@ee-test04 ~]# ltfsee info tapes
        Tape ID   Status        Density       Capacity(GiB)  Free(GiB)  Reclaimable(GiB)   Pool   Library  Address  Drive
        JCA879JC  Write Fenced  LTO6                  2242       2242                 0   pool0   lib0     258      0000078D036C
        JCA839JC  Valid         LTO5                  1327       1325                 0   pool0   lib0     259      001013000652
        JCA840JC  Unavailable   LTO6                     0          0                 0   -       lib0     1030     -
        1N0344JB  Unavailable   LTO6                     0          0                 0   -       lib0     1033     -
        JCA878JC  Unavailable   LTO6                     0          0                 0   -       lib0     1032     -
        JCA838JC  Unavailable   LTO6                     0          0                 0   -       lib0     1027     -
  4. Run the ltfsee recover -s command to show what files are on the tape, if needed.
    [root@ee-test04 ~]# ltfsee recover -s -p pool0 -l lib0 -t JCA879JC
        Scanning GPFS file systems for finding migrated/saved objects in tape JCA879JC.
        Tape JCA879JC has 24 files to be recovered. The list is saved to /tmp/ee-test04.26655.gpfs.recoverlist.
    
        == Contents of /tmp/ee-test04.26655.gpfs.recoverlist ==
        133120 1633360969 0   -- /gpfs/automountdir
        256066 1663596170 0   -- /gpfs/TestData/dir1/dir2/dir3
        238592 1750402892 0   -- /gpfs/locktest2
        256069 1688232767 0   -- /gpfs/TestData/dir1/dir2/emptyfile.txt
        256009 1952164030 0   -- /gpfs/TestData/testdata.005
        256013 1247298962 0   -- /gpfs/TestData/testdata.009
        256055 813378055 0   -- /gpfs/TestData/testdata.008
        256059 1983810896 0   -- /gpfs/TestData/testdata.006
        256064 1283252342 0   -- /gpfs/TestData/testdata.007
        256103 1528248220 0   -- /gpfs/TestData/testdata2.008
        256108 1048476827 0   -- /gpfs/TestData/testdata2.000
        256109 974653174 0   -- /gpfs/TestData/testdata2.001
        256110 964674672 0   -- /gpfs/TestData/testdata2.002
        256302 305576213 0   -- /gpfs/TestData/testdata2.0c3
        256303 876560667 0   -- /gpfs/TestData/testdata2.0c4
        256304 56770306 0   -- /gpfs/TestData/testdata2.0c5
        256305 655002532 0   -- /gpfs/TestData/testdata2.0c6
        256306 1129544792 0   -- /gpfs/TestData/testdata2.0c7
        256053 1749965889 0   -- /gpfs/TestData/dir1/dir2/symlink
        256038 1453943544 0   -- /gpfs/TestData/testdata.001
        256043 1956808976 0   -- /gpfs/TestData/testdata.004
        256050 1666149054 0   -- /gpfs/TestData/testdata.002
        256063 270941016 0   -- /gpfs/TestData/testdata.000
        256067 1320070562 0   -- /gpfs/TestData/testdata.003
  5. Run the ltfsee recover -c command to recover the files that have corresponding migrated stubs in Spectrum Scale.
    [root@ee-test04 ~]# ltfsee recover -c -p pool0 -l lib0 -t JCA879JC
        Scanning GPFS file systems for finding migrated/saved objects in tape JCA879JC.
        Tape JCA879JC has 24 files to be recovered. The list is saved to /tmp/ee-test04.32006.gpfs.recoverlist.
        Bulk recalling files in tape JCA879JC
        GLESL268I(00151): 14 file name(s) have been provided to recall.
        GLESL263I(00207): Recall result: 14 succeeded, 0 failed, 0 duplicate, 0 not migrated, 0 not found.
        Making 24 files resident in tape JCA879JC.
        Changed to resident: 10/24.
        Changed to resident: 20/24.
        Scanning remaining objects migrated/saved in tape JCA879JC.
        Scanning non EE objects in tape JCA879JC.
        Recovery of tape JCA879JC is successfully done. 24 files are recovered. The list is saved to /tmp/ee-test04.32006.gpfs.recoverlist.
  6. If the recovery is successful, run the ltfsee recover -r command to remove the tape safely from the pool.
    [root@ee-test04 ~]# ltfsee recover -r -p pool0 -l lib0 -t JCA879JC
        Scanning GPFS file systems for finding migrated/saved objects in tape JCA879JC.
        Tape JCA879JC has no file to be recovered.
        Removed the tape JCA879JC from the pool pool0.
  7. Run the ltfsee tape move ieslot command to move the tape to an IE slot.
  8. Physically remove the recovered tape from the library.

After the recovery procedure, any files that you want on tape must be migrated.