If there is a failure during the optional rename step of coordinated change master key or coordinated refresh, CSFM629I and CSFM630I messages will be written to the ICSF job log to indicate the reason for the failure.
The rename function uses the IDCAMS processor to perform the actual VSAM data set rename. CSFM629I messages are used to route IDCAMS processor messages to the ICSF job log when the IDCAMS processor fails to perform the rename. The CSFM629I messages contain the reason from the IDCAMS failure. These messages are followed by a CSFM630I message that indicates which data set name failed to be renamed to which new name.
CSF.CSFCKDS
CSF.CSFCKDS.DATA
CSF.CSFCKDS.INDEX
CSF.CSFCKDS.ARC
CSF.CSFCKDS.ARC.DATA
CSF.CSFCKDS.ARC.INDEX
In the case of a failure during rename processing, the coordinated function will attempt to back out and rename the data sets back to their original names. If the back out fails, you may end up with a partially renamed data set. This can be easily corrected by performing an IDCAMS ALTER from JCL.
Whenever a rename failure occurs, scan the ICSF job log of the initiating system for CSFM629I and CSFM630I messages. These messages will indicate which data set part failed during rename and if backout processing was able to rename the data set back to its original name. If backout processing was able to rename back to the original name, check the catalog for the data set name that failed to be used for rename. Most likely you have a conflict with the archive data set name and need to either rename existing data sets in your catalog or choose a different archive name.
If backout processing failed to rename your data set back to the original name, use ISPF to confirm that the data set parts match up with what is reported in the CSFM629I and CSFM630I messages.
For example, if during a coordinated CKDS change master key operation, the active CKDS cluster name is successfully renamed to the archive name, but the data portion fails to be renamed, backout processing begins. If backout processing fails to rename the data set back to the original active CKDS name, ICSF will shut down all instances in the sysplex CKDS cluster because the active CKDS name is only half renamed. In this scenario, the following set of messages may be reported in the ICSF job log.
CSFM618I CKDS DATA SET CSF.CSFCKDS RENAMED TO CSF.CSFCKDS.ARC
CSFM629I IDCAMS SYSTEM SERVICES TIME: 13:35:12 06/07/11
CSFM629I
CSFM629I ALTER CSF.CSFCKDS.DATA -
CSFM629I NEWNAME(CSF.CSFCKDS.ARC.DATA )
CSFM629I IDC3013I DUPLICATE DATA SET NAME
CSFM629I IDC3009I ** VSAM CATALOG RETURN CODE IS 8 - REASON CODE IS IGG0CLE6-8
CSFM629I IDC0532I **ENTRY CSF.CSFCKDS.DATA NOT ALTERED
CSFM629I IDC0001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 8
CSFM629I
CSFM629I IDC0002I IDCAMS PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 8
CSFM630I CKDS RENAME FAILED: CSF.CSFCKDS.DATA TO CSF.CSFCKDS.ARC.DATA
CSFM629I IDCAMS SYSTEM SERVICES TIME: 13:35:18 06/07/11
CSFM629I
CSFM629I ALTER CSF.CSFCKDS.ARC -
CSFM629I NEWNAME(CSF.CSFCKDS )
CSFM629I IDC3013I DUPLICATE DATA SET NAME
CSFM629I IDC3009I ** VSAM CATALOG RETURN CODE IS 8 - REASON CODE IS IGG0CLE6-8
CSFM629I IDC0532I **ENTRY CSF.CSFCKDS.ARC NOT ALTERED
CSFM629I IDC0001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 8
CSFM629I
CSFM629I IDC0002I IDCAMS PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 8
CSFM630I CKDS RENAME FAILED: CSF.CSFCKDS.ARC TO CSF.CSFCKDS
CSFM620I COORDINATED CHANGE-MK MAINLINE PROCESSING FAILED BECAUSE THE ACTIVE DATA SET
CANNOT BE RENAMED TO THE ARCHIVE NAME.
CSFM622I COORDINATED CHANGE-MK PROGRESS: BACKOUT IS BEING DRIVEN BY MAINLINE.
CSFM629I IDCAMS SYSTEM SERVICES TIME: 13:35:24 06/07/11
CSFM629I
CSFM629I ALTER CSF.CSFCKDS.ARC -
CSFM629I NEWNAME(CSF.CSFCKDS )
CSFM629I IDC3013I DUPLICATE DATA SET NAME
CSFM629I IDC3009I ** VSAM CATALOG RETURN CODE IS 8 - REASON CODE IS IGG0CLE6-8
CSFM629I IDC0532I **ENTRY CSF.CSFCKDS.ARC NOT ALTERED
CSFM629I IDC0001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 8
CSFM629I
CSFM629I IDC0002I IDCAMS PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 8
CSFM630I CKDS RENAME FAILED: CSF.CSFCKDS.ARC TO CSF.CSFCKDS
CSFM621I COORDINATED CHANGE-MK BACK OUT PROCESSING FAILED BECAUSE THE ARCHIVE DATA SET
CANNOT BE RENAMED TO THE ACTIVE NAME.
CSFM621I COORDINATED CHANGE-MK BACK OUT PROCESSING FAILED BECAUSE ICSF IS UNABLE TO
RELIABLY RESTORE THE ORIGINAL ACTIVE KDS.
CSFM622I COORDINATED CHANGE-MK PROGRESS: CANCELING CORE WORK.
CSFM616I COORDINATED CHANGE-MK FAILED, RC=0000000C RS= 00000C3E SUPRC= 0000000C SUPRS=
00000C42 FLAGS= C5800000.
CSFU006I CHANGE-MK FEEDBACK: CC=0000000C RSN=00000C3E SUPPRC=0000000C SUPPRSN=00000C42
FLAGS=C5800000.
CSFM308I MEMBER XXX REPORTED REMOVED FROM SYSPLEX GROUP SYSICSF.
CSFM308I MEMBER XXX REPORTED REMOVED FROM SYSPLEX GROUP SYSICSF.
CSFM401I CRYPTOGRAPHY - SERVICES ARE NO LONGER AVAILABLE.
This sequence of messages indicates that the active CKDS name of CSF.CSFCKDS was renamed to CSF.CSFCKDS.ARC. Then, the CSF.CSFCKDS.DATA data portion of the active CKDS failed to be renamed to CSF.CSFCKDS.DATA.ARC because another data set in the catalog was already using this name. At this point, the coordinated CKDS change master key function tried to back out and rename the cluster portion of the CKDS from CSF.CSFCKDS.ARC back to its original name of CSF.CSFCKDS. However the renaming failed because another data set with name CSF.CSFCKDS now exists in the catalog. The result is a half renamed active CKDS which causes ICSF to shut down across the CKDS sysplex cluster.
The first step to resolving this problem is to confirm in ISPF that the following data set names reported in the messages above do exist:
CSF.CSFCKDS.ARC
CSF.CSFCKDS.DATA
CSF.CSFCKDS.INDEX
Once this is confirmed, the next step is to rename the cluster name back to the original name manually by calling IDCAMS ALTER from JCL. Before doing that, the messages above indicate that back out processing already failed to rename the cluster name back because another data set is now using that name. The data set that has taken that name should be renamed to a different name as this name is needed to restore the active CKDS.
Once the cluster name conflict has been resolved, issue IDCAMS ALTER from JCL to rename the CSF.CSFCKDS.ARC cluster name back to the original active CKDS name of CSF.CSFCKDS.
//DEFINE EXEC PGM=IDCAMS,REGION=4M
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
ALTER CSF.CSFCKDS.ARC -
NEWNAME(CSF.CSFCKDS)
/*
ICSF may be restarted on all instances that were previously taken down. Processing should resume as normal and the coordinated CKDS change master key with rename option may be issued again with an archive data set name that does not have a conflict in the catalog.