Recovery of "Critical" status and "Write Fenced" tapes in the inventory

When a tape or tape drive in the IBM Spectrum Archive™ Enterprise Edition inventory encounters a problem, the status of the tapes that are involved can change to Critical status. You can use the ltfsee recover command to help you handle these tapes and recover data from them.

With the ltfsee recover command, migrated files can be recalled from the tape to Spectrum Scale. Other files can be copied to a specified location on Spectrum Scale or local disk.

Note: Check any Spectrum Scale policies that are used for migration. Make any necessary updates to them to remove the tape that has a status of Critical from use.

As part of the recovery operation, the tape that has a status of Critical is removed from the Spectrum Archive Enterprise Edition inventory. Then, the tape drive is made available again. If removal of the tape fails, you must move the tape from the drive by using the appropriate interface for your tape library. In this case, refer to your tape library documentation for details about operating the tape library.

For more information, see the ltfsee recover command.

Typical user scenario for recovering a tape with a "Critical" or "Write Fenced" status

The following scenario is a typical example of encountering and recovering a tape in the Spectrum Archive EE inventory that has a status of "Critical" or "Write Fenced".

An error is encountered for migration, recall, reclaim, import, or export operations, and an SNMP trap notification is received.
A look into ltfsee.log and ltfs.log shows that the error came from writing data.

The ltfsee info tapes command is run, which shows that a tape has a status of "Critical" or "Write Fenced".

[root@ee-test04 ~]# ltfsee info tapes
    Tape ID   Status        Density       Capacity(GiB)  Free(GiB)  Reclaimable(GiB)   Pool   Library  Address  Drive
    JCA879JC  Write Fenced  LTO6                  2242       2242                 0   pool0   lib0     258      0000078D036C
    JCA839JC  Valid         LTO5                  1327       1325                 0   pool0   lib0     259      001013000652
    JCA840JC  Unavailable   LTO6                     0          0                 0   -       lib0     1030     -
    1N0344JB  Unavailable   LTO6                     0          0                 0   -       lib0     1033     -
    JCA878JC  Unavailable   LTO6                     0          0                 0   -       lib0     1032     -
    JCA838JC  Unavailable   LTO6                     0          0                 0   -       lib0     1027     -

Run the ltfsee recover -s command to show what files are on the tape, if needed.

[root@ee-test04 ~]# ltfsee recover -s -p pool0 -l lib0 -t JCA879JC
    Scanning GPFS file systems for finding migrated/saved objects in tape JCA879JC.
    Tape JCA879JC has 24 files to be recovered. The list is saved to /tmp/ee-test04.26655.gpfs.recoverlist.

    == Contents of /tmp/ee-test04.26655.gpfs.recoverlist ==
    133120 1633360969 0   -- /gpfs/automountdir
    256066 1663596170 0   -- /gpfs/TestData/dir1/dir2/dir3
    238592 1750402892 0   -- /gpfs/locktest2
    256069 1688232767 0   -- /gpfs/TestData/dir1/dir2/emptyfile.txt
    256009 1952164030 0   -- /gpfs/TestData/testdata.005
    256013 1247298962 0   -- /gpfs/TestData/testdata.009
    256055 813378055 0   -- /gpfs/TestData/testdata.008
    256059 1983810896 0   -- /gpfs/TestData/testdata.006
    256064 1283252342 0   -- /gpfs/TestData/testdata.007
    256103 1528248220 0   -- /gpfs/TestData/testdata2.008
    256108 1048476827 0   -- /gpfs/TestData/testdata2.000
    256109 974653174 0   -- /gpfs/TestData/testdata2.001
    256110 964674672 0   -- /gpfs/TestData/testdata2.002
    256302 305576213 0   -- /gpfs/TestData/testdata2.0c3
    256303 876560667 0   -- /gpfs/TestData/testdata2.0c4
    256304 56770306 0   -- /gpfs/TestData/testdata2.0c5
    256305 655002532 0   -- /gpfs/TestData/testdata2.0c6
    256306 1129544792 0   -- /gpfs/TestData/testdata2.0c7
    256053 1749965889 0   -- /gpfs/TestData/dir1/dir2/symlink
    256038 1453943544 0   -- /gpfs/TestData/testdata.001
    256043 1956808976 0   -- /gpfs/TestData/testdata.004
    256050 1666149054 0   -- /gpfs/TestData/testdata.002
    256063 270941016 0   -- /gpfs/TestData/testdata.000
    256067 1320070562 0   -- /gpfs/TestData/testdata.003

Run the ltfsee recover -c command to recover the files that have corresponding migrated stubs in Spectrum Scale.

[root@ee-test04 ~]# ltfsee recover -c -p pool0 -l lib0 -t JCA879JC
    Scanning GPFS file systems for finding migrated/saved objects in tape JCA879JC.
    Tape JCA879JC has 24 files to be recovered. The list is saved to /tmp/ee-test04.32006.gpfs.recoverlist.
    Bulk recalling files in tape JCA879JC
    GLESL268I(00151): 14 file name(s) have been provided to recall.
    GLESL263I(00207): Recall result: 14 succeeded, 0 failed, 0 duplicate, 0 not migrated, 0 not found.
    Making 24 files resident in tape JCA879JC.
    Changed to resident: 10/24.
    Changed to resident: 20/24.
    Scanning remaining objects migrated/saved in tape JCA879JC.
    Scanning non EE objects in tape JCA879JC.
    Recovery of tape JCA879JC is successfully done. 24 files are recovered. The list is saved to /tmp/ee-test04.32006.gpfs.recoverlist.

If the recovery is successful, run the ltfsee recover -r command to remove the tape safely from the pool.

[root@ee-test04 ~]# ltfsee recover -r -p pool0 -l lib0 -t JCA879JC
    Scanning GPFS file systems for finding migrated/saved objects in tape JCA879JC.
    Tape JCA879JC has no file to be recovered.
    Removed the tape JCA879JC from the pool pool0.

Run the ltfsee tape move ieslot command to move the tape to an IE slot.
Physically remove the recovered tape from the library.

After the recovery procedure, any files that you want on tape must be migrated.