Flashes (Alerts)
Abstract
IBM has identified an issue in IBM Spectrum Archive Library Edition (LE) 2.4.0.0 through 2.4.3.0, in which a WRITE PERM error for SCSI LOCATE command is treated as if the deferred write error was reported from the tape drive. IBM Spectrum Archive LE incorrectly invalidates the user data blocks on tape from the current tape position at the WRITE PERM error to the end of tape. The affected files are truncated to the 0-byte length on tape or a shorter length than the original, and this may result in undetected data corruption or in deletion of valid data blocks (data loss).
Content
Problem Summary
Problem Determination:
IBM Spectrum Archive stores the files on tape in the file system format, maintains the file system metadata blocks and the user data blocks on tape cartridge, and issues the SCSI commands to the tape drive for writing those blocks to the tape cartridge, or reading them from the tape cartridge. The SCSI LOCATE command is used to change the current position on tape before reading the different portion of tape cartridge or before switching from the read operation to the write operation or vice versa.
When IBM Spectrum Archive Library Edition (LE) receives the WRITE PERM error from the tape drive, it treats the error as the asynchronous error notification for pending writes and the error recovery routine in IBM Spectrum Archive LE attempts to reflect the incomplete writes to filesystem metadata, and invalidates the user data blocks between the current tape position and the end of tape, and marks the failed tape as "DEGRADED" and disables the further writes (read only mode) for that tape. Although this error recovery is still valid for other SCSI commands, it should not be applied to the case when the tape drive reports a WRITE PERM error for the LOCATE command because the current tape position may not be able to tell the failure location. As the result of unexpected truncation of files by the result of the LOCATE command, the previously written files on tape may appear as a 0-byte length file.
Note that the user blocks are still kept on tape at this point.
When IBM Spectrum Archive Library Edition (LE) receives the WRITE PERM error from the tape drive, it treats the error as the asynchronous error notification for pending writes and the error recovery routine in IBM Spectrum Archive LE attempts to reflect the incomplete writes to filesystem metadata, and invalidates the user data blocks between the current tape position and the end of tape, and marks the failed tape as "DEGRADED" and disables the further writes (read only mode) for that tape. Although this error recovery is still valid for other SCSI commands, it should not be applied to the case when the tape drive reports a WRITE PERM error for the LOCATE command because the current tape position may not be able to tell the failure location. As the result of unexpected truncation of files by the result of the LOCATE command, the previously written files on tape may appear as a 0-byte length file.
Note that the user blocks are still kept on tape at this point.
When this happens, LTFS30205I or LTFS30408I message may appear in ltfs.log file followed by LTFS11334I and LTFS11571I messages
LTFS30205I xxx LOCATE xxx returns -xxxxx.
...
LTFS11334I Remove extent : xxxxxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxx-xxxxxxxxxx-xxxxxxxx-x (x, xxxxxxxxxxxx).
...
LTFS11571I State of tape 'xxxxxxXX' in slot 0x10a is changed from 'WRITABLE' to 'WRITE_FENCED'.
...
LTFS11334I Remove extent : xxxxxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxx-xxxxxxxxxx-xxxxxxxx-x (x, xxxxxxxxxxxx).
...
LTFS11571I State of tape 'xxxxxxXX' in slot 0x10a is changed from 'WRITABLE' to 'WRITE_FENCED'.
Note: The log file and its archive files are stored in /var/log directory of the server running IBM Spectrum Archive LE on Linux, and C:\Program Files\IBM\LTFS\log directory on Windows. The log files keep the most recent logs for weeks or a few months depending on the system log setting and the system activity, and the past logs might be already deleted from the system.
Users Affected
This issue may affect customers running, or who had used, IBM Spectrum Archive Library Edition (LE) 2.4.0.0 through 2.4.3.0, when the following conditions are met:
- A tape was marked as "DEGRADED" when the user was running any of those versions.
- The tape failure was caused by WRITE PERM error for LOCATE command.
- A tape was marked as "DEGRADED" when the user was running any of those versions.
- The tape failure was caused by WRITE PERM error for LOCATE command.
Problem Determination:
There are no clear indicators whether this problem had occurred previously or not, other than the read operation returning unexpected data. To determine if there is a problem, perform the Script procedures.
Script Procedure:
Download zero-check-le.README and zero-check-le.py from Fix Central and follow the procedure in the README to identify if the potentially affected tapes are still retained in the system, and to determine whether a file on the tape needs to be recovered before an application reads the file.
To download:
Script Procedure:
Download zero-check-le.README and zero-check-le.py from Fix Central and follow the procedure in the README to identify if the potentially affected tapes are still retained in the system, and to determine whether a file on the tape needs to be recovered before an application reads the file.
To download:
- Go to Fix Central web page.
- Specify "IBM Spectrum Archive Library Edition (LE)" from the Product Selector, and specify "2.4" for Installed Version. Find the files in the latest fix pack.
- Download the zero-check-le.README and zero-check-le.py files.
The tapes that were previously removed from the tape library need to be reinserted before running the script. If you cannot reinsert all the previously removed tapes at once, run the script on a subset of those tapes. Then repeat this until all of the previously removed tapes have been reinserted and processed using the script procedure.
Recommendations:
1. The users running IBM Spectrum Archive Library Edition (LE) 2.4.0.0 through 2.4.3.0 should apply IBM Spectrum Archive Library Edition 2.4.3.1 available from IBM Fix Central.
2. Run the problem determination steps above.
Recommendations:
1. The users running IBM Spectrum Archive Library Edition (LE) 2.4.0.0 through 2.4.3.0 should apply IBM Spectrum Archive Library Edition 2.4.3.1 available from IBM Fix Central.
2. Run the problem determination steps above.
[{"Line of Business":{"code":"LOB26","label":"Storage"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"STZMZN","label":"IBM Spectrum Archive Library Edition"},"ARM Category":[{"code":"a8m0z000000bq6tAAA","label":"Spectrum Archive->Library Edition (LE)"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"2.4.0;2.4.1;2.4.2;2.4.3"}]
Historical Number
2
Was this topic helpful?
Document Information
Modified date:
29 April 2021
UID
ibm16415809