Configuring backup and restore for OpenSearch

Configure IBM Cloud Object Storage (COS) as an OpenSearch snapshot repository and use it to back up and restore indices.

Before you begin

  • Provision an OpenSearch service instance.
  • Ensure you have cluster permissions to create Kubernetes secrets and edit OpenSearch custom resources.
  • Install and configure the OpenShift CLI (oc) and Kubernetes CLI (kubectl).
  • Ensure you have the COS bucket name, access key, and secret key.
  • Ensure you know the OpenSearch admin credentials, the operator namespace (<os-operator-ns>), and the OpenSearch cluster name (<os-test-cluster>).

About this task

OpenSearch supports the S3 repository type for snapshots. IBM Cloud Object Storage provides an S3-compatible API that you can use as a snapshot repository. This task walks you through creating a secret for COS credentials, adding that secret to the OpenSearch keystore, registering a snapshot repository, and taking and restoring snapshots.

Procedure

  1. Create a Kubernetes secret for COS credentials.

    Create a secret that stores the COS access and secret keys used by the S3 client configuration in OpenSearch. 

    
kubectl create secret generic <cos-bucket11-secret> \
  --from-literal=s3.client.bucket11.access_key=<access-key> \
  --from-literal=s3.client.bucket11.secret_key=<secret-key> \
  -n <os-operator-ns>
    Tip: Use a meaningful secret name that matches your S3 client identifier (for example, bucket11).
  2. Add the secret to the OpenSearch keystore.

    Edit the OpenSearchCluster resource and include the secret under spec.general.keystore. 

    
kubectl edit opensearchcluster <os-test-cluster> -n <os-operator-ns>

    Add the following under spec.general:

    
keystore:
  - secret:
      name: <cos-bucket11-secret>
    Important: After you save the changes, the operator reconciles the keystore. Wait until the OpenSearch pods roll or report the updated configuration before proceeding.
  3. Register the COS-backed snapshot repository.

    Use the OpenSearch REST API to register a repository of type s3, pointing to your COS bucket and endpoint. 

    
curl -k -u "<admin credentials>" -X PUT "https://<host>:9200/_snapshot/<ibm_cos_repo>" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "s3",
    "settings": {
      "bucket": "opensearch-s3",
      "endpoint": "https://s3.us-west.cloud-object-storage.test.appdomain.cloud",
      "base_path": "<base directory path>"
    }
  }'

    Successful registration returns a response similar to:

    
{"acknowledged":true}%
    Tip: Replace the bucket, endpoint, and base_path values with your COS configuration. If you use a regional or cross‑region endpoint, make sure it matches your bucket's location.
  4. Take a snapshot.

    Initiate a snapshot and wait for the operation to complete. 

    
curl -k -u "<admin credentials>" -X PUT \
  "https://<host>:9200/_snapshot/<ibm_cos_repo>/<snapshot_name>?wait_for_completion=true"
    Note: If you omit wait_for_completion=true, the API returns immediately and you can track progress in /_cat/snapshots/<ibm_cos_repo>.
  5. Restore from a snapshot.

    Restore data from an existing snapshot. You can restore the full cluster state or specific indices with additional request body options. 

    
curl -k -u "<admin credentials>" -X POST \
  "https://<host>:9200/_snapshot/<ibm_cos_repo>/<snapshot_name>/_restore?wait_for_completion=true"
    Attention: Restoring can overwrite existing indices with the same names. Consider closing or deleting indices before restore, or restore to different index names with rename_pattern and rename_replacement parameters.
  6. Verify repository, snapshots, and restore results.

    Check repository status:

    
curl -k -u "<admin credentials>" \
  "https://<host>:9200/_snapshot/_all"

    List snapshots:

    
curl -k -u "<admin credentials>" \
  "https://<host>:9200/_cat/snapshots/<ibm_cos_repo>?v"

    Verify indices after restore:

    
curl -k -u "<admin credentials>" \
  "https://<host>:9200/_cat/indices?v"

Results

IBM Cloud Object Storage is configured as the OpenSearch snapshot repository. You can create and manage snapshots and restore data as required.

What to do next

  • Set up a scheduled snapshot process using your preferred automation (for example, a CronJob or external scheduler).
  • Monitor snapshot health and repository size usage regularly.
  • Rotate credentials and update the keystore secret when keys change.
Troubleshooting:
  • If repository registration fails, verify that the COS endpoint, bucket name, and keystore credentials are correct, and ensure the OpenSearch nodes have network egress to the COS endpoint.
  • If snapshots fail to complete, check the OpenSearch logs for S3 client errors and confirm that the bucket IAM policy allows PutObject, GetObject, ListBucket, and DeleteObject operations.
Security considerations:
  • Limit access to the credentials secret via namespace RBAC.
  • Use TLS for all connections to the COS endpoint and validate certificates.
  • Apply lifecycle policies to the bucket to control retention and costs.