Cleaning up indices

By default, OpenSearch retains all data indefinitely. If you need to remove old data—whether to free up space or to prevent Business Performance Center from accessing outdated information—you can use the REST API to perform the cleanup.

About this task

The following procedure deletes all data and alarm states, but keeps the user dashboards defined in Business Performance Center. When cleaning up indices, avoid deleting the index. Instead, purge its contents. Purging refers to deleting the data within an index while retaining the index itself.

Procedure

  1. Select indices to clean up.
    Note: Indices with active in their names, for example, those containing summaries for in-progress operations, will regenerate as soon as new events are added to the active summary.
  2. Login to OpenShift®:
    kubectl login <cluster>
  3. Export NAMESPACE=production # namespace where Business Automation Insights is deployed:
    export NAMESPACE=production # namespace where BAI is deployed
    OPENSEARCH_URL="https://$(kubectl get routes opensearch-route -o jsonpath="{.spec.host}" -n "$NAMESPACE")"
    OPENSEARCH_USERNAME=$(kubectl get secret/opensearch-admin-user -o json -n "$NAMESPACE" | jq -r '.data|keys[0]')
    OPENSEARCH_PASSWORD=$(kubectl extract secret/opensearch-admin-user --keys="$OPENSEARCH_USERNAME" --to=- -n "$NAMESPACE" 2>/dev/null
  4. Purge the index content by running the following curl command. It will delete all documents from the specified index ${INDEX}. Iterate over all relevant indices as needed:
    curl -X POST -u "$OPENSEARCH_USERNAME:$OPENSEARCH_PASSWORD" "https://$OPENSEARCH_URL/${INDEX}/_delete_by_query?pretty" -H 'Content-Type: application/json' --insecure -d'
    { 
      "query": { "match_all": {} }
    }
    '
  5. Purge the Business Performance Center alerts by running the following curl command:
    INDEX='icp4ba-bai-store-alert*'
    curl -X POST -u"$OPENSEARCH_USERNAME:$OPENSEARCH_PASSWORD""https://$OPENSEARCH_URL/${INDEX}/_delete_by_query?pretty"-H'Content-Type: application/json'--insecure-d'
    { 
      "query": { "match_all": {} }
    }
    '