Example of recovery using volume backup

In this example, we simulated the recovery from the loss of a volume by performing a volume restore before the forward recovery process. Backout-failed UOWs were the result of the I/O errors that occurred when the volume failed.

Note: It is important to ensure that CICS® cannot retry the shunted UOWs when the volume is restored, until after the forward recovery work is complete. This is done by quiescing the volume before it is restored, as described under Volume recovery procedure using CFVOL QUIESCE (step 1).

Many of the steps in this second example are the same as those described under the Example of recovery using data set backup, and are listed here in summary form only.

We simulated the volume failure using the MVS™ command.
```
ROUTE *ALL,VARY 4186,OFFLINE,FORCE
```
We stopped the I/O errors by closing the files that were open against failed data sets. In our example, file F04DENDB was open against data set RLSADSW.FV04D.DATAENDB and file F04DCTRL was open against data set RLSADSW.FV04D.TELLCTRL.
Because the failed data sets were restored from the same volume, there was no need to delete the catalog entries for these data sets.
Before restoring the failed volume, we quiesced the volume to ensure that CICS could not access the restored data sets. by issuing the command:
```
VARY SMS,CFVOL(9S4186),QUIESCE
```
In this example, for volume serial 9S4186, the command produced the message:
```
IGW462I DFSMS CF CACHE REQUEST TO QUIESCE VOLUME 9S4186 IS ACCEPTED
```
We confirmed that the volume was quiesced by issuing the MVS command:
```
DISPLAY SMS,CFVOL(9S4186)
```
which confirmed that the volume was quiesced with the message:
```
IGW531I DFSMS CF VOLUME STATUS
VOLUME = 9S4186
DFSMS VOLUME CF STATUS = CF_QUIESCED
VOLUME 9S4186 IS NOT BOUND TO ANY DFSMS CF CACHE STRUCTURE
```
We simulated the volume restore for this example by using the MVS VARY command to bring the volume back online:
```
ROUTE *ALL,VARY 4186,ONLINE
```
Because the volume was quiesced, attempts to open files on this volume failed, with messages such as the following:
```
DFHFC0500 ADSWA02A RLS OPEN of file F04DENDB failed. VSAM has
          returned code X'0008' in R15 and reason X'00C6'.
```
The impact of the recovery process is greater if there are inflight tasks updating RLS mode files. To minimize the impact, you are recommended at this point to quiesce all data sets that are being accessed in RLS mode.
We terminated the SMSVSAM servers with the MVS command:
```
ROUTE *ALL,VARY SMS,SMSVSAM,TERMINATESERVER
```
When all SMSVSAM servers were down, we deleted the IGWLOCK00 lock structure with the MVS command:
```
VARY SMS,SMSVSAM,FORCEDELETELOCKSTRUCTURE
```
We restarted the SMSVSAM servers with the MVS command:
```
ROUTE *ALL,VARY SMS,SMSVSAM,ACTIVE
```
CICS was informed during dynamic RLS restart about the data sets for which it must perform lost locks recovery. CICS issued messages such as the following to inform you that lost locks recovery was being performed on one or more data sets:
```
+DFHFC0555 ADSWA04A One or more data sets are in lost locks status.
           CICS will perform lost locks recovery.
```
If we had quiesced data sets before terminating the servers, this is the point at which we would unquiesce those data sets before proceeding.

If there were many data sets in lost locks it would take some time for lost locks recovery to complete. It may be necessary to explicitly open files which suffer open failures during lost locks recovery.
At this point it was possible that there were data sets on the restored volume which did not require forward recovery. In order to make these data sets available, we needed to re-allow access to the volume. Before doing this, however, we first had to quiesce the data sets that still required forward recovery, thus transferring the responsibility of preventing backouts from SMSVSAM to CICS. In our example, we quiesced our two data sets using the CEMT commands:
```
SET DSN(RLSADSW.VF04D.DATAENDB) QUIESCED
SET DSN(RLSADSW.VF04D.TELLCTRL) QUIESCED
```
When we were sure that all data sets requiring forward recovery were quiesced, we used the following MVS command to allow access to the restored volume:
```
VARY SMS,CFVOL(9S4186),ENABLE
```
The previous command produced the following message:
```
IGW463I DFSMS CF CACHE REQUEST TO ENABLE
              VOLUME 9S4186 IS COMPLETED.
              DFSMS CF VOLUME STATUS = "CF_ENABLED"
```
At this point, all data sets were available for new work except the two quiesced data sets on the restored volume. We recovered these using CICS VSAM Recovery for z/OS.
All CICS regions were automatically notified when CICS VSAM Recovery for z/OS processing for each data set was complete, and each data set was automatically unquiesced by CICS VSAM Recovery for z/OS to allow the backout shunted UOWs to be retried.
After all backout shunted UOWs were successfully retried, the recovery was complete and we re-enabled the recovered data sets for general access on each CICS region using the CEMT commands:
```
SET FILE(F04DENDB) ENABLED
SET FILE(F04DCTRL) ENABLED
```
Finally, we used the SHCDS command LISTSUBSYS(ALL) to confirm that no CICS region had lost locks recovery outstanding, indicating that recovery was complete.