High availability manual recovery procedures

Manual recovery procedures for common problems.

The Batches / ICLs pages still show the duplicate reviewed status as in-progress

After an IBM® MQ pod failure while transmissions are being loaded, the following states are seen on the Processing Batches / ICLs or Inbound Batches / ICLs page.
  • One or more batches (ICLs) has the in-progress status in the duplicate reviewed column.
  • The captured, loaded, Image Quality Analysis, duplicate detected, repaired, reviewed, and ISN allocated columns have a checkmark.
  • The billed, balanced, keyed, and adjusted columns are blank.
To recover, use the Control Center to do the following steps.
  1. Identify the names of the transmissions.
    1. On the Processing Batches / ICLs page, get the inbound transmission ID for the batches (ICLs) that are still in progress.
    2. On the Inbound Transmissions page, note the name of the transmission in the original name column.
  2. Go to the Manage units of work page.
  3. Select the Do not use start and end times checkbox.
  4. Click Show unit of Work.
  5. Click Re-post Completion for each transmission that you identified in step 1.
  6. Verify that the The completion message for unit of work File Name was re-posted successfully. message was displayed for each transmission.

Work-in-progress records for type duplicate detection show a messaging error on the Service manager page even though the status of the associated batch (ICL) is complete

After an IBM MQ pod was re-created while transmissions were being loaded, work-in-progress records of type duplicate detection were found in the messaging error state. To determine whether this problem is occurring, use the Control Center to display the work-in-progress records with messaging errors.
  1. Go to the Service manager page.
  2. Select the applicable business day.
  3. Filter the work-in-progress records that are shown on the page. Select the following filtering criteria and then click Go.
    • Select messaging error for the state column.
  4. Notice the remaining work-in-progress records where the type column is Duplicate Detection and the description column is DEFAULT DUP PROFILE.

To recover, use these filtered results on the Service manager page. Select all the work-in-progress records and then choose Restart from the Selection menu.

After a IBM MQ problem while loading, the adjusted column is in the in-progress state even though the status of the batch (ICL) is complete

After an IBM MQ problem such as a pod failure or queue issues, the following states for some inbound batches (ICLs) are seen on the Adjustment management page.
  • The batch (ICL) might show in-progress for the adjusted column, but the status and the ready for distribution columns are marked complete.
To recover, use the Control Center to do the following steps.
  1. Go to the Adjustment management page.
  2. Select the correct business day.
  3. Filter the batches (ICLs) that are shown on the page. Select the following filtering criteria and then click Go.
    • Select not adjusted for the adjusted column.
    • Select balanced for the balanced column.
  4. Select the batches (ICLs) that need to be adjusted.
  5. Choose Adjust selected Batches / ICLs from the Selection menu.
  6. On the Perform adjustment dialog box, select every checkbox and click Start.
  7. Ensure that the adjustment submitted successfully message is displayed.
  8. If the adjustment was successful, the batches (ICLs) now show the complete status on the Inbound Batches / ICLs page.

The status of some batches (ICLs) is processing and the Service manager page shows a messaging error for duplicate detection

This problem might occur after an IBM MQ pod failed while transmissions were being loaded. To determine whether this problem is occurring, use the Control Center to do the following steps.
  1. Go to the Inbound Batches / ICLs page and see whether the following states are seen on the page.
    • One or more batches (ICLs) has the processing status, and ready for distribution is set to no for the batch (ICL).
    • The captured, loaded, Image Quality Analysis, repaired, reviewed, and ISN allocated columns have a checkmark.
    • The duplicate detected, duplicate reviewed, billed, balanced, keyed, and adjusted columns are blank.
  2. Go to the Service manager page.
  3. Select the applicable business day.
  4. Filter the work-in-progress records that are shown on the page. Select the following filtering criteria and then click Go.
    • Select messaging error for the state column.
  5. Notice that the value in the type column for the work-in-progress records is duplicate detection.
To recover, use the Control Center to do the following steps.
  1. Go to the Service manager page.
  2. Select the applicable business day.
  3. Filter the work-in-progress records that are shown on the page. Select the following filtering criteria and then click Go.
    • Select duplicate detection for the type column.
    • Select messaging error for the state column.
  4. Select all the work-in-progress records and then choose Restart from the Selection menu.

The status of an inbound batch (ICL) remains as processing and is not balanced

This problem can occur after a Gateway pod fails while transmissions are being loaded. To determine whether this problem is occurring, use the Control Center to do the following steps.
  1. Go to the Inbound Batches / ICLs page and see whether the following states are seen on the page.
    • One or more batches (ICLs) has the processing status, and ready for distribution is set to no for the batch (ICL).
To recover, use the Control Center to do the following steps.
  1. Go to the Adjustment management page.
  2. Select the correct business day.
  3. Filter the batches (ICLs) that are shown on the page. Select the following filtering criteria and then click Go.
    • Select the value both for the adjusted column.
    • Select not balanced for the balanced column.
  4. Select the batches (ICLs) that need to be adjusted.
  5. Choose Adjust selected Batches / ICLs from the Selection menu.
  6. On the Perform adjustment dialog box, select every checkbox and click Start.
  7. Ensure that the adjustment submitted successfully message is displayed.
  8. If the adjustment was successful, the batches (ICLs) now show the complete status on the Inbound Batches / ICLs page.

Batches (ICLs) are missing from the Transmissions and Batch / ICL pages and the batch (ICL) files are in the error directory

This problem can occur after any of the following things happen while transmissions are being loaded.
  • A Db2® takeover or failure.
  • An IBM MQ failure.
  • A Gateway pod failure.
You can use the Control Center to check whether the runtime-processing, inbound, or error directories contain transmissions by doing the following steps.
  1. Go to the Inbound Batches / ICLs page and see whether one or more batches (ICLs) are missing.
  2. Go to the Physical Transmissions page.
  3. Choose the correct business day.
  4. Filter the transmissions that are shown on the page. Select the following filtering criteria and then click Go.
    • Select error for the status column.
  5. The page should show the same transmissions that are in the error directory.
To recover, use the Control Center to do the following steps.
  1. Go to the Physical Transmissions page and select the transmissions that are in error status.
  2. Choose Restart from the Selection menu.
  3. Verify that the restart occurred by checking the following things.
    • The transmissions have a status of processed on the Physical Transmissions page.
    • The transmissions appear on the Inbound Transmissions page.
    • All transmissions have a status of complete on the Inbound Batches / ICLs page.
    • The transmissions that were restarted are no longer in the error folder.

The status of some batches (ICLs) is processing and the Service manager page shows messaging for duplicate detection

This problem can occur after a pod fails while transmissions are being loaded. To determine whether this problem is occurring, use the Control Center to do the following steps.
  1. Go to the Inbound Batches / ICLs page and see whether the following states are seen on the page.
    • One or more batches (ICLs) has the processing status, and ready for distribution is set to no for the batch (ICL).
  2. Go to the Service manager page.
  3. Select the applicable business day.
  4. Filter the work-in-progress records that are shown on the page. Select the following filtering criteria and then click Go.
    • Select messaging for the state column.
  5. Notice that the value in the type column for the work-in-progress records is duplicate detection.
To recover, use the Control Center to do the following steps.
  1. Go to the Service manager page.
  2. Select the applicable business day.
  3. Filter the work-in-progress records that are shown on the page. Select the following filtering criteria and then click Go.
    • Select messaging for the state column.
  4. Select all the work-in-progress records and then choose Restart from the Selection menu.

The status of the batches (ICLs) is processing and the Service manager page shows error for duplicate detection

This problem can occur after a pod fails while transmissions are being loaded. To determine whether this problem is occurring, use the Control Center to do the following steps.
  1. Go to the Inbound Batches / ICLs page and see whether the following states are seen on the page.
    • One or more batches (ICLs) has the processing status, and ready for distribution is set to no for the batch (ICL).
  2. Go to the Service manager page.
  3. Select the applicable business day.
  4. Filter the work-in-progress records that are shown on the page. Select the following filtering criteria and then click Go.
    • Select error for the state column.
  5. Notice that the value in the type column for the work-in-progress records is duplicate detection.
To recover, use the Control Center to do the following steps.
  1. Go to the Service manager page.
  2. Select the applicable business day.
  3. Filter the work-in-progress records that are shown on the page. Select the following filtering criteria and then click Go.
    • Select error for the state column.
  4. Select all the work-in-progress records and then choose Restart from the Selection menu.

The adjusted column shows in-progress or error

This problem can happen after a Db2 takeover or failure occurs while transmissions are being loaded. To determine whether this problem is occurring, use the Control Center to do the following steps.
  1. Go to the Inbound Batches / ICLs page and see whether the following states are seen on the page.
    • All batches (ICLs) have a status of complete and ready for distribution is set to complete. But, the adjusted column shows in-progress or error for one or more batches (ICLs).
To recover, use the Control Center to do the following steps.
  1. Go to the Service manager page.
  2. Select the applicable business day.
  3. Filter the work-in-progress records that are shown on the page. Select the following filtering criteria and then click Go.
    • Select error for the state column.
  4. Select all the work-in-progress records and then choose Restart from the Selection menu.
If some batches (ICLs) on the Processing Batches / ICLs page are still in-progress after the restart, do adjustment.
  1. Go to the Adjustment management page.
  2. Select the correct business day.
  3. Filter the batches (ICLs) that are shown on the page. Select the following filtering criteria and then click Go.
    • Select not adjusted for the adjusted column.
    • Select the value both for the balanced column.
  4. Select the batches (ICLs) that need to be adjusted.
  5. Choose Adjust selected Batches / ICLs from the Selection menu.
  6. On the Perform adjustment dialog box, select every checkbox and click Start.
  7. Ensure that the adjustment submitted successfully message is displayed.
  8. If the adjustment was successful, the batches (ICLs) now show the complete status on the Processing Batches / ICLs page.

The Service manager page shows a work-in-progress record in messaging state for settlement

Sometimes work-in-progress records with the type settlement were found in the messaging state on the Service manager page. To determine whether this problem is occurring, use the Control Center to do the following steps.
  1. Go to the Inbound Batches / ICLs page and see whether the following states are seen on the page.
    • All batches (ICLs) have the complete status and ready for distribution is set to complete.
    • All columns have a checkmark.
  2. Go to the Service manager page.
  3. Select the applicable business day.
  4. Filter the work-in-progress records that are shown on the page. Select the following filtering criteria and then click Go.
    • Select messaging for the state column.
  5. The page shows one or more work-in-progress records that have a type that is set to settle, a description that is settlement, and a state that is messaging.
To recover, use the Control Center to do the following steps.
  1. Go to the Service manager page.
  2. Select the applicable business day.
  3. Filter the work-in-progress records that are shown on the page. Select the following filtering criteria and then click Go.
    • Select messaging for the state column.
  4. Select all the work-in-progress records and then choose Restart from the Selection menu.
  5. Notice that the remaining work-in-progress records have a type that is set to settle and a description that is settlement.

Duplicate reviewed is in-progress for one or more batches (ICLs)

After a Db2U pod failure while transmissions are being loaded, the following states are seen on the Processing Batches / ICLs page.
  • One or more batches (ICLs) has the processing status, and ready for distribution is set to no for the batch (ICL).
  • The captured, loaded, Image Quality Analysis, duplicate detected, repaired, reviewed, and ISN allocated columns have a checkmark.
  • Duplicate reviewed is set to in-progress.
  • The billed, balanced, keyed, and adjusted columns are blank.
To recover, use the Control Center to do the following steps.
  1. Identify the names of the transmissions.
    1. On the Processing Batches / ICLs page, get the inbound transmission ID for the batches (ICLs) that are still in progress.
    2. On the Inbound Transmissions page, note the name of the transmission in the original name column.
  2. Go to the Manage units of work page.
  3. Select the Do not use start and end times checkbox.
  4. Click Show unit of Work.
  5. Click Re-post Completion for each transmission that you identified in step 1. If this action is not available for a transmission, try the Duplicate reviewed is in-progress for one or more batches (ICLs) - part 2 recovery procedure.
  6. Verify that the The completion message for unit of work File Name was re-posted successfully. message was displayed for each transmission.
  7. If duplicate reviewed is still set to in-progress on the Processing Batches / ICLs page, try the Duplicate reviewed is in-progress for one or more batches (ICLs) - part 2 recovery procedure.

Duplicate reviewed is in-progress for one or more batches (ICLs) - part 2

After an Db2U pod failure while transmissions are being loaded, the following states are seen on the Processing Batches / ICLs page.
  • One or more batches (ICLs) has the processing status, and ready for distribution is set to no for the batch (ICL).
  • The captured, loaded, Image Quality Analysis, duplicate detected, repaired, reviewed, and ISN allocated columns have a checkmark.
  • Duplicate reviewed is set to in-progress.
  • The billed, balanced, keyed, and adjusted columns are blank.
To recover, use the Control Center to do the following steps.
  1. Check the state of the transmission that corresponds with each batch (ICL) by using the Processing Batches / ICLs page. Get the IDs of the transmissions from the Inbound Transmission ID column for the batches (ICLs) that are still processing.
  2. Go to the Inbound transmissions page.
  3. Select the transmissions that were identified in step 1.
  4. Use the show locked column to determine whether the transmission is locked. If it is locked, unlock it by selecting the transmission and doing the Unlock action.
  5. If the batch (ICL) is still in the processing state, go to the Processing Batches / ICLs page. Select the batches (ICLs) that have in-progress for duplicate reviewed.
  6. Use the State Management action to set the duplicate detected state to on for the batch (ICL). This action changes the duplicate detected column to display a checkmark because the state is now set to on. It might also cause other states to complete and a checkmark to display in their columns.
  7. Use the State Management action to set the following states to on for the batch (ICL), as needed.
    • Duplicate reviewed
    • Keyed
    • Adjusted
  8. Verify that both the status and the ready for distribution status were changed to complete.