BBOT

BBOT0001D: OTS ENCOUNTERED HARDENED DATA LOSS DURING RESTART OF {0}. REPLY "CONTINUE" OR "CANCEL"

Explanation

RRS returned ATR_HARDENED_DATA_LOST from the Begin_Restart service (ATRIBRS).

Action

Reply CONTINUE or CANCEL. If the reply is CONTINUE, Object Transaction Service (OTS) might not be able to resolve all transactions that are in doubt. Replying CANCEL terminates the control region. Consult the RRS documentation for information on resolving this condition.

BBOT0002E: "{0}" IS NOT A VALID REPLY.

Explanation

Reply to a previous message was not valid. The previous message is reissued.

Action

Issue a valid reply to the reissued message.

BBOT0003W: The Transaction with URID {0} has timed out after processing for more than {1} seconds. The transaction will be rolled back. The transaction XID is {2}.

Explanation

A global transaction in which the CORBA Object Transaction Service was involved has exceeded its maximum processing time. The transaction was marked for rollback only. Any further method dispatches in this transaction will cause a CORBA::TRANSACTION_ROLLEDBACK exception.

Action

Examine the application for a possible coding error. The application might explicitly set the transaction timeout value to a value less than or equal to the value of the transaction_maximumTimeout environment variable. If a timeout was not explicitly set, it defaults to the value in the transaction_defaultTimeout environment variable.

BBOT0004E: RRS SERVICE "{0}" FAILED WITH RETURN CODE={1}.

Explanation

The indicated resource recovery service has failed with the given hexadecimal return code.

Action

See return codes for resource recovery services in the MVS Programming: Resource Recovery book for additional information.

BBOT0005I: UR {0} IS NOW IN THE {1} STATE WITH RRS

Explanation

This message is issued after a UR goes in-Doubt with RRS. It is only issued after this debug aid has been explicitly enabled with a modify command. This is an internal programming error.

Action

If the problem persists, additional information might be available if you search for the message ID on the following Web sites: WebSphere Application Server Support page: https://ibm.biz/BdztgV WebSphere Application Server for z/OS Support page: https://ibm.biz/Bdqd4J .

BBOT0006I: The transaction with URID {0} and XID {1} has completed heuristically.

Explanation

One or more resource managers involved in the global transaction whose ID is identified in the message were assigned an outcome that was different from the outcome of the global transaction. For example, if the outcome of the transaction was commit, some resources that were involved in the transaction have been rolled back.

Action

This message shows the transaction identifier for which the heuristic outcome occurred. Use the ATRFPCMN dialog facility to review the resource recovery services (RRS) transaction logs and determine which resources are inconsistent. Follow the procedures for that particular resource manager to restore or correct the state of those resources.

BBOT0007I: The transaction with URID {0} and XID {1} may have completed heuristically.

Explanation

One or more resource managers involved in the global transaction whose ID is identified in the message might be assigned an outcome that was different from the outcome of the global transaction.

Action

This message shows the transaction identifier for which the heuristic outcome might have occurred. Use the ATRFPCMN dialog facility to review the resource recovery services (RRS) transaction logs and determine whether any resources are inconsistent. Follow the procedures for that particular resource manager to restore or correct the state of those resources. Also, if this condition occurs when transactional optimizations are used, RRS might not be aware of the transactional outcome, because commit responsibilities are delegated to the single resource manager. In this situation, the outcome of the single resource manager is unknown and a hazard condition exists. To determine the outcome, follow documented procedures for the resource manager identified in the message to determine the outcome or to correct any resources involved in the global transaction shown in this message.

BBOT0008I: TRANSACTION SERVICE RESTART INITIATED ON SERVER {0}

Explanation

The transaction services component has initiated transaction restart and recovery with the resource recovery services (RRS).

Action

No action is required.

BBOT0009I: TRANSACTION SERVICE RESTART UR STATUS COUNTS FOR SERVER {0}: IN-BACKOUT={1}, IN-DOUBT={2}, IN-COMMIT={3}

Explanation

The transaction services component has received restart units of recovery from the resource recovery services (RRS). The counts indicate the number of units of work that had states when the server was stopped previously such that some recovery action is required during this restart. The IN-BACKOUT count specifies the number that were in the process of backing out. These must undergo recovery to restore resources that they modified to their previous consistent state. The IN-DOUBT count specifies the number that were interrupted between phase 1 and phase 2 of the commit process. These must undergo recovery to ensure that resources that they modified do not become available until their InDoubt status is resolved. The IN-COMMIT count specifies the number that had started, but not completed, phase 2 of the commit process. These must undergo recovery to complete the commit process.

Action

No action is required.

BBOT0010I: TRANSACTION SERVICE RESTART AND RECOVERY ON SERVER {0} IS COMPLETE

Explanation

The transaction services restart process has completed. All units of recovery have been resolved.

Action

No action is required.

BBOT0011I: SERVER {0} IS COLD STARTING WITH RRS

Explanation

During the resource recovery services (RRS) log name exchange process, RRS indicated that no previous log name was set. When restart is initiated, no incomplete RRS units of recovery will be returned.

Action

No action is required.

BBOT0012I: SERVER {0} IS WARM STARTING WITH RRS

Explanation

During the resource recovery services (RRS) log name exchange process, RRS indicated that a log name was previously set. When restart is initiated, some incomplete RRS units of recovery might be returned.

Action

No action is required.

BBOT0013I: TRANSACTION SERVICE RESTART AND RECOVERY ON SERVER {0} IS COMPLETE. THE SERVER IS STOPPING.

Explanation

The transaction services restart process has completed. All units of recovery have been resolved.

Action

No action is required.

BBOT0014I: TRANSACTION SERVICE RECOVERY PROCESSING FOR RRS URID {0} IN SERVER {1} IS COMPLETE.

Explanation

The transaction service restart processing for the unit of recovery identifier (URID) shown in the message has completed.

Action

No action is required.

BBOT0015D: OTS UNABLE TO RESOLVE ALL INCOMPLETE TRANSACTIONS FOR SERVER {0}. REPLY CONTINUE OR TERMINATE.

Explanation

The transaction service restart processing for the server running in restart and recovery mode was unable to resolve all incomplete transactions in the configured amount of time. This is generally due to a communications failure with a another server that was involved in the transaction. It is possible that unresolved transactions will prevent a successful restart with Resource Recovery Services (RRS) on the server"s configured system.

Action

If the server should continue its attempts to resolve incomplete transactions for the duration specified by the transaction_recoveryTimeout environment variable, reply CONTINUE. If you do not want the server to continue its attempts to resolve incomplete transactions, reply TERMINATE to stop the server. Information about the unresolved transactions can be found in the server"s job output. This information can be used to resolve transactions using the RRS panels.

BBOT0016I: TRANSACTION SERVICE RESTART AND RECOVERY FOR SERVER {0} IS NOT COMPLETE. THE SERVER IS STOPPING DUE TO OPERATOR REPLY.

Explanation

The transaction services restart processing has not completed within the configured time. The operator has indicated (by reply) that the server should no longer attempt recovery. Some units of recovery have not been completed and forgotten.

Action

None.

BBOT0017I: TRANSACTION SERVICE RESTART AND RECOVERY FOR SERVER {0} IS CONTINUING DUE TO OPERATOR REPLY.

Explanation

The transaction services restart processing has not completed within the configured time. The operator has indicated (by reply) that the server should continue to attempt transaction completion for an additional duration time specified by the transaction_recoveryTimeout environment variable.

Action

None.

BBOT0018I: TRANSACTION SERVICE RESTART AND RECOVERY FOR SERVER {0} IS STILL PROCESSING {1} INCOMPLETE UNIT(S) OF RECOVERY.

Explanation

Action

No action is required.

BBOT0019W: UNABLE TO RESOLVE THE OUTCOME OF TRANSACTION BRANCH DESCRIBED BY: URID: {0} XID FORMATID: {1} XID GTRID: {2} XID BQUAL: {3} BECAUSE THE OTS RECOVERY COORDINATOR FOR SERVER {4} ON HOST {5}:{6} COULD NOT BE REAACHED.

Explanation

While in restart and recovery mode, the application server could not contact the recovery coordinator for the transaction branch with the XID provided. The host name and port were obtained from the Interoperable Object Reference (IOR) for the recovery coordinator. If the IOR points to a WebSphere/390 object, the server name is also written.

Action

Ensure that the server and host can be reached by this application server. If connectivity cannot be restored, you can use the transaction information in this message to resolve the outcome of the transaction manually with the resource recovery services (RRS) ISPF panels or the ATRSRV macro. Note that resolving the transaction manually can result in mixed transaction outcomes across resource managers and servers.

BBOT0020W: UNABLE TO PROVIDE THE SUBORDINATE OTS RESOURCE IN SERVER {0} ON HOST {1}:{2} WITH THE OUTCOME OF THE TRANSACTION DESCRIBED ON THIS SERVER BY: URID: {3} XID FORMATID: {4} XID GTRID: {5} XID BQUAL: {6} BECAUSE THIS SERVER HAS BEEN UNABLE TO RESOLVE THE OUTCOME WITH A SUPERIOR NODE.

Explanation

While in restart and recovery mode, the application server could not contact the recovery coordinator for the transaction branch with the XID provided. Because the outcome could not be determined, this server could not provide a transaction resolution to the resources registered with the transaction coordinator on this server for the transaction branch with the XID information in this message.

Action

No action is required.

BBOT0021W: UNABLE TO {0} THE SUBORDINATE OTS RESOURCE IN SERVER {1} ON HOST {2}:{3} FOR THE TRANSACTION DESCRIBED ON THIS SERVER BY: URID: {4} XID FORMATID: {5} XID GTRID: {6} XID BQUAL: {7} OR ANOTHER RESOURCE INVOLVED IN THIS UNIT OF RECOVERY BECASUE ONE OR MORE RESOURCES COULD NOT BE REACHED OR HAVE NOT YET REPLIED.

Explanation

While in restart and recovery mode, the application server could not successfully commit or roll back the resource involved in the transaction branch described by the XID information in this message. The outcome of the transaction has been determined, but it cannot be completed until all subordinate OTS resources have been told the outcome.

Action

Ensure that the server and host can be reached by this application server. If connectivity cannot be restored, you can use the transaction information in this message to resolve the outcome of the transaction for the subordinate nodes manually. The CORBA Object Transaction Service (OTS) interests in the resource recovery services (RRS) unit of recovery with the unit of recovery identifier (URID) shown can be removed with the RRS ISPF panels or the ATRSRV macro.

BBOT0022W: UNABLE TO FORGET THE TRANSACTION WITH HEURISTIC OUTCOME DESCRIBED ON THIS SERVER BY: URID: {1} XID FORMATID: {2} XID GTRID: {3} XID BQUAL: {4} BECASUE THE SUPERIOR COORDINATOR FOR SERVER {5} ON HOST {6}:{7} HAS NOT INVOKED FORGET ON THE REGISTERED RESOURCE.

Explanation

While in restart and recovery mode, the application server was not told to forget an heuristic outcome for a transaction it was involved in. A transaction that is not forgotten is not complete, because the resource recovery services (RRS) unit of recovery (UR) with the unit of recovery identifier (URID) shown is still active.

Action

Ensure that the server and host in this message can successfully connect to this server. If connectivity cannot be restored, or the superior does not know of the heuristic outcome, you can use the RRS ISPF panels or ATRSRV macro to remove the CORBA Object Transaction Service (OTS) interests in the UR with the URID shown.

BBOT0023A: WEBSPHERE INITIALIZATION CAN NOT PROCEED UNTIL RRS IS AVAILABLE.

Explanation

WebSphere Application Server for z/OS detected that RRS was not available. WebSphere Application Server for z/OS requires RRS to be active and will not proceed with its initialization until RRS is available.

Action

Start RRS. Once RRS is available, WebSphere Application Server for z/OS will continue with its initialization.

BBOT0024A: RRS HAS BECOME UNAVAILABLE. WEBSPHERE MUST BE RESTARTED.

Explanation

WebSphere Application Server for z/OS was notified that RRS is no longer available on this system and has terminated processing. RRS and WebSphere Application Server for z/OS must be restarted before new work will be accepted.

Action

WebSphere Application Server for z/OS must be restarted on the system where RRS became unavailable.

BBOT0025D: OTS HAS ENCOUNTERED A LOG DATA MISMATCH. REPLY CONTINUE IF THIS IS EXPECTED OR TERMINATE IF UNEXPECTED.

Explanation

WebSphere Application Server for z/OS has recovery work to do, but the versions of the RRS log and XA partner log do not match. In order for recovery to complete successfully, these logs must be in sync. It is possible that one or both of these logs were deleted, or restored to a previous state that is not consistent. The RRS log is stored as a logstream defined at installation time. The XA partner log is defined through the admin console. It is also possible that the controller failed to initialize on the previous restart, which would not allow the XA partner log to sync up with the RRS logstream.

Action

If the log data mismatch is expected, reply CONTINUE to the WTOR. If the mismatch is unexpected, reply CANCEL to terminate the controller and determine the cause of the mismatch. Note that by replying CONTINUE, recovery will complete but there is a chance that some resources will be left in an inconsistent state.

BBOT0026I: TRANSACTION SERVICE RESTART AND RECOVERY FOR SERVER {0} IS STILL PROCESSING AN UNKNOWN NUMBER OF XA TRANSACTIONS.

Explanation

The transaction services restart processing is still trying to resolve some incomplete restart units of recovery in restart and recovery mode. This message is issued at regular intervals to show the progress of the transaction recovery process. Because each XA resource manager maintains its own logs, it is impossible to tell how many units of recovery need to be resolved until each resource manager is contacted.

Action

No action is required.

BBOT0027I: TRANSACTION SERVICE ON SERVER {0} IS WAITING FOR HA RECOVERY TO TERMINATE ON ANOTHER SERVER

Explanation

Transactional recovery for this server is being performed on a peer server using the high availability (HA) recovery framework. This server must wait for the HA recovery process to stop before it can take over the transactional logs for this server.

Action

No action is required. The peer server has been notified that it must stop recovery.

BBOT0028I: HA RECOVERY UNABLE TO COMPLETE FOR SERVER {0} AFTER {1} RETRIES. REPLY CONTINUE OR TERMINATE.

Explanation

The transaction service recovery processing was unable to resolve all incomplete transactions in the configured amount of retries. This is generally due to a communications failure with a another server that was involved in the transaction.

Action

If the server should continue its attempts to resolve incomplete transactions for the duration specified by the transaction_heuristicRetryLimit environment variable, reply "CONTINUE." If the server should not continue its attempts to resolve incomplete transactions, a reply of "TERMINATE" will stop the recovery process. Information about the unresolved transactions can be found in the server"s job output. This information can be used to resolve transactions using the RRS panels.

BBOT0029I: TRANSACTION SERVICE RECOVERY FOR SERVER {0} ON HOST SERVER {1} IS CONTINUING DUE TO OPERATOR REPLY. CONFIGURED RECOVERY RETRY LIMIT: {2}.

Explanation

The transaction services recovery processing has not completed within the configured amount of retries. The operator has indicated (by reply) that recovery process should continue to attempt transaction completion for an additional amount of times specified by the transaction_heuristicRetryLimit environment variable.

Action

None.

BBOT0030I: TRANSACTION SERVICE RECOVERY FOR SERVER {0} ON HOST SERVER {1} IS TERMINATING DUE TO OPERATOR REPLY.

Explanation

The transaction services recovery processing has been terminated because the operator has indicated (by reply) that recovery process should stop. Some Units of Recovery have not been completed.

Action

None.

BBOT0031I: TRANSACTION ON SERVER {0} AND DESCRIBED BY URID: {1} XID FORMATID: {2} XID GTRID: {3} XID BQUAL: {4} HAS HEURISTICALLY BEEN SET TO COMMIT AFTER {5} COMPLETION ATTEMPTS ON HOST SERVER {6}

Explanation

The transaction service"s recovery processing has not completed within the configured amount of retries. The transaction"s outcome has been heuristically set to commit according to the setting of "Heuristic completion direction" in the transaction service configuration.

Action

No action is required.

BBOT0032I: TRANSACTION ON SERVER {0} AND DESCRIBED BY URID: {1} XID FORMATID: {2} XID GTRID: {3} XID BQUAL: {4} HAS HEURISTICALLY BEEN SET TO ROLLBACK AFTER {5} COMPLETION ATTEMPTS ON HOST SERVER {6}

Explanation

The transaction service"s recovery processing has not completed within the configured amount of retries. The transaction"s outcome has been heuristically set to rollback according to the setting of "Heuristic completion direction" in the transaction service configuration.

Action

No action is required.

BBOT0033I: TRANSACTION ON SERVER {0} AND DESCRIBED BY URID: {1} XID FORMATID: {2} XID GTRID: {3} XID BQUAL: {4} HAS HEURISTICALLY BEEN SET TO MANUAL AFTER {5} COMPLETION ATTEMPTS ON HOST SERVER {6}

Explanation

The transaction service"s recovery processing has not completed within the configured amount of retries. The transaction"s outcome has been heuristically set to manual according to the setting of "Heuristic completion direction" in the transaction service configuration.

Action

Access the RRS panels to view and resolve any pending transactions. */

BBOT0034I: TRANSACTION SERVICE RECOVERY FOR SERVER {0} ON HOST {1} WAS TERMINATED BECAUSE RECOVERY IS ALREADY BEING PERFORMED ON ANOTHER SERVER.

Explanation

The transaction services recovery processing was stopped because recovery for the failed server is already taking place on another server. The transaction service could not obtain the resources needed because they are being held by another server that is attempting perform recovery on the same failed server.

Action

BBOT0035I: TRANSACTION SERVICE RECOVERY FOR SERVER {0} ON HOST {1} WILL BE TERMINATED BECAUSE THE FAILED SERVER HAS BEEN RESTARTED.

Explanation

The transaction services recovery processing will be stopped because the server for which recovery is taking place has been restarted. If recovery was not completed on the peer server, it will continue during server restart.

Action

No action is required.

BBOT0036E: A CORBA::NO_PERMISSION EXCEPTION WAS RAISED WHILE DRIVING {0} ON A BRANCH OF THE TRANSACTION DESCRIBED BY URID: {1} XID FORMATID {2} XID GTRID {3} XID BQUAL {4} WITH SERVER {5} ON HOST {6}:{7}. THE OPERATION WILL BE RETRIED.

Explanation

The transaction service attempted to drive a transactional object residing on the server, host, and port specified in the message. The security credentials supplied on the call were rejected by the remote server.

Action

The security policy of the remote server might have changed after the transaction was started. In this situation, update the security policy on the remote server to temporarily allow this transaction to complete. The operation will retry automatically.

BBOT0037I: WEBSPHERE BACKED OUT URID {0} AT TASK TERMINATION. THE RETURN CODE FROM ATREND IS {1}.

Explanation

A task in the address space is terminating with an in-flight UR. WebSphere Application Server ended the UR, and reports the return code from the ATREND service.

Action

If the return code from ATREND is not zero, manual action might be necessary to return transactional resources to a consistent state. Return code descriptions are found in the book titled "MVS: Programming Resource Recovery". If necessary, query the RRS archive log to determine the outcome of the UR, and which resources had expressed an interest in the UR.