transparent cloud tiering status description

The following table describes the various statuses and their description that is associated with the health of cloud services running on each node in the cluster.

Table 1. cloud services status description
Entity Status Description Comments
TCT Account Status Not configured The transparent cloud tiering is installed, but the account is not configured or deleted.

Run the mmcloudgateway account create command to create a cloud provider account.

Active The cloud provider account that is configured with transparent cloud tiering service is active.

N/A

Configured The cloud provider account is configured with transparent cloud tiering, but the service is down.

Run the mmcloudgateway service start command to resume the cloud gateway service.

Unreachable The cloud provider access point URL is unreachable because of network issues or when the cloud provider access point URL is down.

Ensure that the cloud provider is online. Check whether the network is reachable between the cloud provider and transparent cloud tiering. Also, check the DNS settings.

For more information, check the trace messages and error log.

TCT Account Status invalid_csv_endpoint_URL The reason might be because of an HTTP 404 Not Found error.

Check whether the configured cloud provider URL is valid.

malformed_URL The cloud provider access point URL is malformed.

Check whether the following conditions are met:

  • Cloud provider URL is valid.
  • Cloud provider URL has a proper legal protocol, such as HTTP or HTTPS.
  • Cloud provider URL has space or any special characters that cannot be parsed.
no_route_to_csp The response from the cloud storage access point is invalid.

Check whether the following conditions are met:

  • DNS and firewall settings are configured.
  • Network is reachable between transparent cloud tiering and the cloud provider.
connect_exception The connection is refused remotely by the CSAP.

Check whether the following conditions are met:

  • Network is reachable between transparent cloud tiering and the cloud provider.
  • Cloud provider URL is valid.
  • Cloud provider receives connections.
TCT Account Status socket_timeout Timeout occurred on a socket while it was connecting to the cloud provider.

Check whether network is reachable between transparent cloud tiering and the cloud provider.

invalid_cloud_config Transparent cloud tiering refuses to connect to the CSAP.

Check whether the cloud object store is configured correctly.

For a Swift cloud provider, check whether both Keystone and Swift provider configuration are proper. Also, check whether Swift is reachable over Keystone.

credentials_invalid The transparent cloud tiering service fails to connect to CSAP because of failed authentication.

Check whether the access key and secret key are valid. Also, check whether the username and password are correct.

mcstore_node_network_down The network of the transparent cloud tiering node is down.

Check whether the network interface on the transparent cloud tiering node is proper and is able to communicate with public and private networks.

ssl_handshake_exception The CSAP fails due to an unknown SSL handshake error.

Check whether the following conditions are met:

  • Cloud provider supports secured communication and is properly configured with certificate chain.
  • The provided cloud provider URL is secure (HTTPS).
  • A secure connection to cloud provider is established by running the openssl s_client -connect <cloud provider ipaddress>: <secured_port> command on a transparent cloud tiering node.
TCT Account Status SSL handshake certificate exception Transparent cloud tiering failed to connect to the CSAP because of an untrusted server certificate chain or certificate is invalid.

Check whether the following conditions are met:

  • The server certificate is not expired, still valid, and the DER is properly encoded.
  • A self-signed or internal CA signed certificate is added to Transparent cloud tiering truststore. Use –server-cert-path option to add a self-signed certificate.
  • The server certificate is valid by using the openssl x509 -in <server_cert> -text -noout command. Check the Validity section and ensure that the certificate is not expired and still valid.
SSL handshake sock closed exception. Transparent cloud tiering fails to connect to the CSAP because the remote host closed connection during the handshake.

Check whether the following conditions are met:

  • A secure network connection is established and that secure port is reachable.
  • A secured connection is established to the cloud provider by running the openssl s_client -connect <cloud_provider_ipaddress>: <secured_port> command on the Transparent cloud tiering node.
SSL handshake bad certificate exception Transparent cloud tiering fails to connect to the CSAP because the server certificate does not exist the truststore.

Ensure that a self-signed or internal CA-signed certificate is properly added to the Transparent cloud tiering truststore. Use the –server-cert-path option to add a self-signed certificate.

TCT Account Status SSL handshake invalid path certificate exception Transparent cloud tiering fails to connect to the CSAP because it is unable to find a valid certification path.

Check whether the following conditions are met:

  • A proper self-signed or internal CA-signed certificate is added to the Transparent cloud tiering truststore. Use the –server-cert-path option to add a self-signed certificate.
  • A client (Transparent cloud tiering ) and server (cloud provider) certificate are signed with same certificate authority or configured with same self-signed certificate.
  • Proper cloud provider certificates are returned when a load balancer or firewall is present.
  • A valid certificate chain is returned from the cloud provider by running the openssl s_client -connect <cloud_provider_ipaddress>: <secured_port> command on the Transparent cloud tiering node.
SSL handshake failure exception Transparent cloud tiering fails to connect to the CSAP provider because it might not negotiate the required level of security.

Ensure that the cloud provider supports TLSv1.2 protocol and TLSv1.2 enabled cipher suites.

SSL handshake unknown certificate exception Transparent cloud tiering fails to connect to the CSAP because of an unknown certificate.

Ensure that a proper self-signed or internal CA-signed certificate is added to the Transparent cloud tiering truststore. Use the –server-cert-path option to add a self-signed certificate.

SSL key exception Transparent cloud tiering fails to connect to the CSAP because of bad SSL key or misconfiguration.

Check whether the following conditions are met:

  • The SSL configuration on cloud provider is proper.
  • The Transparent cloud tiering truststore, /var/MCStore/.mcstore.jceks, is not corrupted. If the /var/MCStore/.mcstore.jceks is corrupted, then remove it and restart the server. This action replaces the /var/MCStore/.mcstore.jceks from the CCR file.
TCT Account Status SSL peer unverified exception Transparent cloud tiering fails to connect to the CSAP because its identity is not verified.

Check whether the following conditions are met:

  • The cloud provider has a valid SSL configuration in place.
  • The cloud provider supports TLSv1.2 protocol and TLSv1.2 enabled cipher suites.
  • A secure connection is in place to the cloud provider by using the openssl s_client -connect <cloud_provider_ipaddress>: <secured_port> command on the Transparent cloud tiering node.
SSL protocol exception Transparent cloud tiering failed to connect to the cloud provider because of an error in the operation of the SSL protocol.

For more information, check trace messages and error logs.

SSL exception Transparent cloud tiering failed to connect to the cloud provider because of an error in the SSL subsystem.

For more information, check trace messages and error logs.

SSL no certificate exception Transparent cloud tiering failed to connect to the cloud provider because a certificate was not available.

For more information, check trace messages and error logs.

TCT Account Status SSL not trusted certificate exception. Transparent cloud tiering failed to connect to the cloud provider because it might not locate a trusted server certificate.

For more information, check trace messages and error logs.

SSL invalid algorithm exception Transparent cloud tiering failed to connect to the cloud provider because of invalid or inappropriate SSL algorithm parameters.

For more information, check trace messages and error logs.

SSL invalid padding exception Transparent cloud tiering failed to connect to the cloud provider because of invalid SSL padding.

For more information, check trace messages and error logs.

SSL unrecognized message Transparent cloud tiering failed to connect cloud provider because of an unrecognized SSL message.

For more information, check trace messages and error logs.

Bad request Transparent cloud tiering failed to connect to the cloud provider because of a request error.

For more information, check trace messages and error logs.

Precondition failed Transparent cloud tiering failed to connect to the cloud provider because of a precondition failed error.

For more information, check trace messages and error logs.

Default exception The cloud provider account is not accessible due to an unknown error.

For more information, check trace messages and error logs.

TCT Account Status Container creation failed The cloud provider container creation failed. The cloud provider account might not be authorized to create the container.

For more information, check trace messages and error logs. Also, check whether the account creates problems in Transparent cloud tiering issues.

Time skew The time that is observed on the transparent cloud tiering service node is not in sync with the time on the target cloud provider.

Change the transparent cloud tiering service node timestamp to be in sync with the NTP server and rerun the operation.

Server error Transparent cloud tiering failed to connect to the cloud provider because of a cloud provider server error (HTTP 503) or the container size reached max storage limit.

For more information, check trace messages and error logs.

Internal directory not found Transparent cloud tiering failed because one of its internal directories is not found.

For more information, check trace messages and error logs.

Database corrupted The database of Transparent cloud tiering service is corrupted.

For more information, check trace messages and error logs. Run the mmcloudgateway files rebuildDB command to repair when any issues are found.

TCT File System Status Not configured Transparent cloud tiering installed, but the file system is not configured or it was deleted.

Run the mmcloudgateway filesystem create command to configure the file system.

Configured The transparent cloud tiering is configured with a file system. N/A
TCT Server Status Stopped transparent cloud tiering service is stopped by CLI command or stopped itself due to some error.

Run the mmcloudgateway service start command to start the cloud gateway service.

Suspended The cloud service was suspended manually.

Run the mmcloudgateway service start command to resume the cloud gateway service.

Started The cloud gateway service is running. N/A
Not configured transparent cloud tiering was either not configured or its services were never started.

Set up the transparent cloud tiering and start the service.

Security rkm down The remote key manager that is configured for transparent cloud tiering is not accessible.

For more information, check trace messages and error logs.

lkm down The local key manager who is configured for transparent cloud tiering is either not found or corrupted.

For more information, check trace messages and error logs.

Cloud Storage Access Point ONLINE Cloud storage access point that is configured with transparent cloud tiering service is active.

The state when the cloud storage access point is reachable, and credentials are valid.

OFFLINE The cloud storage access point URL is unreachable because it is down or has network issues.

unreachable

The cloud provider end-point URL is unreachable.

OFFLINE The reason might be because of HTTP 404 Not Found error.

invalid_csp_endpoint_url

The specified end-point URL is invalid.

OFFLINE The cloud storage access point URL is malformed.

malformed_url

The URL is malformed.

OFFLINE The response from cloud storage access point is invalid. no_route_to_csp

There is no route to the CSP, which indicates a possible firewall issue.

OFFLINE The connection is refused remotely by the cloud storage access point. connect_exception

There is a connection exception when you connect.

OFFLINE Timeout occurs on a socket while it connects to the cloud storage access point URL. SOCKET_TIMEOUT

The socket times out.

Cloud Storage Access Point OFFLINE transparent cloud tiering refuses to connect to the cloud storage access point. invalid_cloud_config

There is an invalid cloud configuration, check the cloud.

OFFLINE The transparent cloud tiering service fails to connect cloud storage access point because the authentication fails. credentials_invalid

The credentials that are provided during addCloud are no longer valid (including if the password is expired).

OFFLINE The network of transparent cloud tiering node is down. mcstore_node_network_down

The network is down on the mcstore node.

OFFLINE The cloud storage access point status fails due to an unknown SSL handshake error. ssl_handshake_exception

There is an SSL handshake exception.

OFFLINE transparent cloud tiering fails to connect cloud storage access point because of an untrusted server certificate chain. ssl_handshake_cert_exception

There is an SSL handshake certificate exception.

Cloud Storage Access Point OFFLINE transparent cloud tiering fails to connect cloud storage access point because the remote host closed connection during handshake. ssl_handshake_sock_closed_exception

There is an SSL handshake socket closed exception.

OFFLINE transparent cloud tiering fails to connect cloud storage access point because of a bad certificate. ssl_handshake_bad_cert_exception

There is an SSL handshake bad certificate exception.

OFFLINE transparent cloud tiering fails to connect cloud storage access point because it is unable to find a valid certification path. ssl_handshake_invalid_path_cert _exception

There is an SSL handshake invalid path certificate exception.

OFFLINE transparent cloud tiering fails to connect to the cloud storage access point because it might not negotiate the required level of security. ssl_handshake_failure_exception

There is an SSL handshake failure exception.

OFFLINE transparent cloud tiering fails to connect cloud storage access point because of an unknown certificate. ssl_handshake_unknown_cert _exception

There is an SSL handshake unknown certificate exception.

Cloud Storage Access Point OFFLINE transparent cloud tiering fails to connect cloud storage access point because of a bad SSL key or misconfiguration. ssl_key_exception

There is an SSL key exception.

OFFLINE transparent cloud tiering fails to connect cloud storage access point because its identity is not verified. ssl_peer_unverified_exception

Check whether the following conditions are met:

  • The transparent cloud tiering has a valid SSL configuration in place.
  • The cloud provider supports TLSv1.2 protocol and TLSv1.2 enabled cipher suites.
  • A secured connection to the cloud provider is in place by using the openssl s_client -connect <cloud_provider_ipaddress>: <secured_port> command on the transparent cloud tiering node.
OFFLINE transparent cloud tiering fails to connect to the cloud storage access point because of an SSL protocol operation error. ssl_protocol_exception

Ensure that the Cloud Provider SSL configuration supports TLSv1.2 protocol.

Cloud Storage Access Point OFFLINE transparent cloud tiering fails to connect to the cloud storage access point because of an SSL subsystem error. ssl_exception

Check whether a secured connection to the cloud provider is in place by using the openssl s_client -connect <cloud_provider_ipaddress>: <secured_port> command on the transparent cloud tiering.

For more information, check error log and traces to find further cause and take necessary actions, as this is generic error.

OFFLINE transparent cloud tiering fails to connect to the cloud storage access point because there is no certificate available. ssl_no_cert_exception

Check whether the following conditions are met:

  • A self-signed or internal root CA certificate is properly added to the transparent cloud tiering truststore.
  • Add a self-signed certificate or internal root CA certificate by using the –server-cert-path option.
OFFLINE transparent cloud tiering fails to connect to the cloud storage access point because the server certificate is not trusted. ssl_not_trusted_cert_exception
  • If the cloud provider is using a self-signed certificate, make sure it is properly added to the transparent cloud tiering truststore.
  • Use the –server-cert-path option to add a self-signed certificate.
  • If the cloud provider is using a third-party CA signed certificate, make sure that the certificate chain is properly established on the cloud provider.
  • Run this command on the transparent cloud tiering node: openssl s_client -connect <cloud_provider_ipaddress>: <secured_port>
Cloud Storage Access Point OFFLINE transparent cloud tiering fails to connect cloud storage access point because of invalid or inappropriate SSL algorithm parameters. ssl_invalid_algo_exception

Check whether the JRE security file (java.security) is installed under /opt/ibm/MCStore/jre/lib/security got updated to an unsupported SSL algorithm or invalid key length.

OFFLINE transparent cloud tiering fails to connect cloud storage access point because of invalid SSL padding. ssl_invalid_padding_exception

Make sure that server SSL configuration uses TLSv11.2 enabled cipher suites.

Cloud Storage Access Point OFFLINE transparent cloud tiering fails to connect cloud storage access point because of an unrecognized SSL message. ssl_unrecognized_msg
Check the following:
  • Check whether the cloud provider URL is secured (https)
  • Check whether the cloud provider supports SSL communication.
OFFLINE transparent cloud tiering fails to the connect cloud storage access point because of a request error. bad_request
Check the following:
  • Check whether the cloud provider URL is valid
  • Check whether the cloud provider is working well through other available tools
OFFLINE transparent cloud tiering fails to connect to the cloud storage access point because of a precondition failed error. precondition_failed

Check whether the cloud provider is responding to requests through other available tools and make sure that the resource is accessible.

Retry the operation once the server is able to serve the requests.

OFFLINE The cloud storage access point account is not accessible due to unknown error. default

Some unchecked exception is caught.

OFFLINE The cloud provider container creation fails. CONTAINER_CREATE_FAILED

The container creation failed.

OFFLINE The cloud provider container creation fails because it exists. CONTAINER_ALREADY_EXISTS

The container creation failed.

Cloud Storage Access Point OFFLINE The cloud provider container creation fails because it exceeds the maximum limit. BUCKET_LIMIT_EXCEEDED

The container creation failed.

OFFLINE The cloud provider container does not exist. CONTAINER_DOES_NOT_EXIST

The cloud operations failed.

OFFLINE The time observed on the transparent cloud tiering service node is not in sync with the time observed on target cloud storage access point. TIME_SKEW
OFFLINE transparent cloud tiering fails to connect cloud storage access point because the cloud storage access point server error or container size reaches the maximum storage limit. SERVER_ERROR
OFFLINE transparent cloud tiering fails because one of its internal directories is not found. The internal directory is not found and shows the internal_dir_notfound message.
Cloud Storage Access Point OFFLINE transparent cloud tiering failed because the resource address file is not found.

The resource address file is not found and shows the RESOURCE_ADDRESSES_FILE_NOT_FOUND message.

OFFLINE The database of transparent cloud tiering service is corrupted.

The transparent cloud tiering service is corrupted and shows the db_corrupted message.

OFFLINE The remote key manager that is configured for transparent cloud tiering is not accessible.

The SKLM server is not accessible. This condition is valid only when ISKLM is configured.

OFFLINE The local key manager who is configured for transparent cloud tiering is either not found or corrupted.

The local .jks file is not found or corrupted. This action is valid only when local key manager is configured.

OFFLINE The reason might be because of an HTTP 403 Forbidden error.

The action is forbidden.

OFFLINE Access denied due to authorization error.

Access is denied.

Cloud Storage Access Point OFFLINE The file system of transparent cloud tiering service is corrupted.

The file system is corrupted.

OFFLINE The directory of transparent cloud tiering service is corrupted.

There is a directory error.

OFFLINE The key manager that is configured for transparent cloud tiering is either not found or corrupted.

There is a key manager error.

OFFLINE transparent cloud tiering failed because its container pair root directory not found.

The container pair root directory is not found.

  transparent cloud tiering service has too many retries internally.

There are too many retries.

Cloud Service ENABLED The cloud service is enabled for cloud operations.

The Cloud service is enabled.

DISABLED The cloud service is disabled.

The Cloud service is disabled.

TCT Service STOPPED The cloud gateway service is down and might not be started.

The server is stopped abruptly, for example, JVM crash.

SUSPENDED The cloud gateway service is suspended manually.

The TCT service is stopped intentionally.

STARTED The cloud gateway service is up and running.

The TCT service is started up and running.

  The cloud gateway check returns an unknown result.

The TCT service status has an unknown value.

  Attempt to restart the cloud gateway process.

The TCT service is restarted.

NOT_CONFIGURED The Transparent cloud tiering was not configured or its service was never started.

The TCT service is recently installed.

Container Pair Status ONLINE The container pair set is configured with an active transparent cloud tiering service.

The state when the container pair set is reachable.

OFFLINE transparent cloud tiering failed to connect to the container pair set because of a directory error.

Connection failure due to directory_error or TCT directory error.

  transparent cloud tiering failed to connect to the container pair set because the container does not exist.

container_does_not_exist

The cloud container cannot be found but the cloud connection is active.

  transparent cloud tiering failed to connect to the cloud provider because of precondition failed error.

For more information, check trace messages and error logs.

  transparent cloud tiering failed to connect because resource address file is not found.

For more information, check trace messages and error logs.

  transparent cloud tiering failed to connect because one of its internal directories is not found.

For more information, check trace messages and error logs.

  transparent cloud tiering failed because one of its internal directories is not found.

For more information, check trace messages and error logs.

  transparent cloud tiering failed to connect cloud storage access point because of cloud storage access point service unavailability error.

For more information, check trace messages and error logs.

  transparent cloud tiering failed to connect cloud storage access point because of a bad certificate.

For more information, check trace messages and error logs.