AFM to cloud object storage issues
The following table lists the common questions in the AFM to cloud object storage.
| Question | Answer or Resolution |
|---|---|
| What can be done if the AFM to cloud object storage relationship in the Unmounted state? |
The relationship is in the Unmounted state because of the buckets on a cloud object storage are not accessible or the gateway can connect to the endpoint but cannot see the buckets and cannot connect. Check that buckets have correct configuration and keys. Use the mmafmctl filesystem getstate command to check the fileset state. The fileset state is transient. When the network issue or bucket issues are resolved, the state becomes Active. You need not to use any commands to change the state. |
| Why an error message is displayed while creating the AFM to cloud object storage relationship? | The No keys for bucket <Bucket_Name> is set for server <Server_name> error message is displayed while creating the AFM to cloud object storage relationship. This error occurs because correct keys are not set for a bucket or no keys are set. Set the keys correctly for the bucket before the relation creation. |
| Why operations are requeued on the gateway node? | An IBM Storage Scale cluster supports special characters. But, when a cloud object storage does not support object or bucket names with special characters, they are requeued. Each cloud object storage provider must have some limitations for bucket and objects names. For more information about the limitations, see the documentation that is provided by cloud object storage providers. |
| What must be done when the AFM to cloud object storage relationship is disconnected? | A primary gateway cannot connect to a cloud object storage endpoint. Check the endpoint configuration and network connection between a cluster and the cloud object storage. |
| What can be done when fileset space limit is approaching? | Callbacks such as lowDiskSpace or noDiskSpace can be set to
confirm that the space or fileset or file system is approaching. Provide a
storage and allocate more storage to the pools that are defined. |
| What can be done when messages in the queue are requeued because of cloud object storage is full? | When the provisioned space on a cloud object storage is full, messages can be requeued on a gateway node. Provide more space on the cloud object storage and run the mmafmctl resumeRequeued command so that the requeued messages are run again. |
| What can be done if there are servers on the mmafmtransfer command but everything looks normal? | When the objects are synchronized to a cloud object storage, the mmafmtransfer command-related waiters can be seen especially for large objects, or when the application is creating or accessing multiple objects at a faster pace. |
| Read seems to be stuck or inflight for a long time. What should be done? | Check the status of the fileset by using mmafmctl getstate command to see whether the fileset is in the Unmounted state. Check for network errors. |
| How does the ls command claim the inodes after metadata eviction? | When metadata is evicted, any operation that requires metadata, for example, the ls command reclaims the metadata. |
| Operations fail with the No buffer space available error message, but the mmdiag --memory command shows that memory is available. What should be done? | Increase the afmHardMemThreshold value. |
| Why some of directories are not replicated? | Empty directories are not replicated. There is no concept of directory in a cloud object storage, directory names are prefixes to the objects. |
| Why does the synchronous keys setting fail? | Sometime, the configuration file is locked for security purpose. Because of this setting, many keys on some directories might fail at a time. These keys can be set again. |
| What can be done if E_PERM or error 13 occurs when you transfer an object? |
AFM to cloud object storage might get E_PERM or error 13 from a cloud object storage on an object if the cloud object storage has such error response. For example, for the signature mismatch error, see the AWS troubleshooting guide. An AFM to COS fileset tries to transfer again these objects to the cloud object storage, and the objects might be requeued on gateway nodes. |