Monitoring fileset states for AFM to cloud object storage
An AFM to cloud object storage can have different states. You can get these states by using the mmafmctl filesystem getstate command according to the relation status it has with the cloud object storage endpoints.
To view the current cache state, run the
mmafmctl filesystem getstatemmafmctl filesystem getstate -j cache_filesetSee the following table for the explanation of the cache state:
| AFM to cloud object storage fileset state | Condition | Description | Healthy or Unhealthy | Administrator's action |
|---|---|---|---|---|
| Inactive | The fileset is created. | A AFM to cloud object storage fileset is created, or operations were not initiated on the cluster after the last daemon restart. | Healthy | None |
| FlushOnly | Operations are queued. | Operations have not started to flush. | Healthy | This is a temporary state and should move to Active when a write is initiated. |
| Active | The fileset cache is active. | The fileset is ready for an operation. | Healthy | None |
| Dirty | The fileset is active. | The pending changes in the fileset are not played on the could object storage. | Healthy | None |
| Recovery | The fileset is accessed after primary gateway failure or shutdown and restart of gateway node or cluster. | A new gateway is taking over a fileset as primary gateway after the old primary gateway failed, or if there are some operations in queue and gateway or cluster is restarted. | Healthy | None |
| QueueOnly | The fileset is running some operation. | Operations such as recovery, resync, failover are being run, and operations are being queued and not flushed. | Healthy | This is a temporary state. |
| Disconnected | Primary gateway cannot connect to a cloud object storage endpoint. | When a gateway node of the fileset cannot connect to a cloud object storage endpoint. | Unhealthy | Check endpoint configuration and network connection between a cluster and a cloud object storage. |
| Unmounted | The buckets on a cloud object storage are not accessible. | The gateway can connect to the endpoint but cannot see the buckets and cannot connect. | Unhealthy | Check that buckets exist with right configuration and keys. |
| Dropped | Recovery operation failed. | The local file system is full, space is not available on the fileset, or case of a policy failure during recovery. | Unhealthy | Fix the issue and access the fileset to retry recovery. Provide more space for the fileset. |
| Dropped | A fileset with active queue operations is forcibly unlinked. | All queued operations are being de-queued, and the fileset remains in the Dropped state and moves to the Inactive state when the unlinking is complete. | Healthy | This is a temporary state. |
| Dropped | In the middle of an operation if the buckets or its configuration on a cloud object storage are changed. | When the relation is set and working, the buckets on a cloud object storage are changed. | Unhealthy | Relink the fileset after buckets and its configuration is changed to its previous state. |
| Dropped | During recovery or normal operation | If gateway queue memory is exceeded, the queue can get dropped. The memory has to be increased to accommodate all requests and bring the queue back to the Active state. | Unhealthy | Increase the afmHardMemThreshold value. |
| Expired | The RO mode relation that is configured to expire. | An event that occurs automatically after prolonged disconnection when the RO fileset contents are not accessible. | Unhealthy | Fix the errant network or bucket changes. |
| Stopped | Replication stopped on fileset. | The fileset stops sending changes to the gateway node. This state is used during planned downtime. | Unhealthy | After planned downtime, run mmafmctl <fs> start -j <fileset> to start sending changes or modification to the gateway node and continue replication. |