Payload storage causing a data center failure

Edit online

A payload storage failure occurred in one of the data centers.

Symptoms

A failed disk prevents uploads and downloads only to that data center. The other data center can still receive and process files if the disk is functioning correctly.

Causes

Payload storage failures can result from incidences such as network outages and hardware failures.

Environment

Windows, AIX, or Linux.

Diagnosing the problem

  1. Check IBM® Control Center for a notification with the error code, GMST00001E.
  2. Check IBM Control Center for a red line connecting a Global Mailbox server to the storage service within the Data center view.
  3. Check the Server component - storage page within IBM Control Center for a status of Down.
  4. Check for the following errors in the messages.log file in the Global Mailbox Admin nodes of the data center experiencing the failure:

    CBXRN0005E: An error occurred while replicating file from replication server ... errorDesc=Source: No such file or directory

    Storage bucket could not be found for: payloadId=[...] segmentId=[...] segmentFileName=...]. Check storage mount.

Resolving the problem

No files can be processed in the failed data center, so traffic must be rerouted to the surviving data center until the problem is corrected. Refer to the following topics for information about how to recover from a payload storage failure with minimal impact on your business operations.