Premigrating Business Automation Insights data to OpenSearch
Premigration is an optional step that is recommended for migrating a high volume of data. Premigration reduces Business Automation Insights downtime as it can be run before you upgrade a CP4BA deployment. Premigration can be run while a CP4BA deployment is running.
Before you begin
About this task
It is recommended to test document migration in a batch of between 100,000 and 500,000. Then use this insight to calculate how long it should take to migrate all of your data given the throughput of your test. Depending on the time, you can decide whether you want to split the premigration of data into multiple batches with the premigration procedures. The time that it takes to complete the migration is related to the volume of data and the performance of the resources (CPU, memory, network, disk).
- Strategy 1: Migration without premigration
-
If document volumes are low, all indexes can be migrated without running the premigration procedures before Business Automation Insights is shut down.
- Strategy 2: Migration with premigration in one batch
- If data volumes allow for the container to process it in one batch, you can migrate your
immutable documents in one go by running the
es-os-migration-script.sh
script on all premigration indexes at once. Shut down Business Automation Insights, and then migrate the remaining documents. - Strategy 3: Migration with premigration in multiple batches
- If data volumes are large, it is necessary to split into multiple migration executions by using
the premigration procedures. You can migrate your immutable documents in chunks by running multiple
batches of the
es-os-migration-script.sh
script on the premigration indexes. Shut down Business Automation Insights, and then migrate the remaining documents.Note: Running the premigration procedures with multiple batches gives the chance to back up the migrated data, and take maintenance actions on the cluster between cycles.
The following diagram shows the three strategies that you can choose. It highlights the volume of Elasticsearch (ES) immutable documents that can be transferred to OpenSearch (OS) in green while event processing remains online (premigration). All three strategies require Business Automation Insights to be shut down to be able to migrate the mutable documents offline, as indicated by the red arrow. Strategy 1 migrates all the documents while Business Automation Insights is offline

es-os-migration-script.sh
script and its description.
Options | Description |
---|---|
-dryrun |
List of indexes of Elasticsearch and displays the dry-run steps. |
-doc_count |
List all indexes of Elasticsearch and OpenSearch with document count, and exit. |
-include=<comma separated indices> |
List of indexes to include. |
-exclude=<comma separated indices> |
List of indexes to exclude. |
-include_regex=<regex pattern> |
List of regex patterns to include indexes. |
-exclude_regex=<regex pattern> |
List of regex patterns to exclude indexes. |
-startdate=<start date> |
Start date for data migration (format: 'YYYY-MM-DDTHH:MM:SS') |
-enddate=<end date> |
End date for data migration (format: 'YYYY-MM-DDTHH:MM:SS') |
-timestamp_key=<name> |
Key for date values (default: 'timestamp') |
-delete |
Delete OpenSearch indexes |
logfile |
Optional: Log file to save the migration summary. |
--help |
Displays usage details. |
Procedure
What to do next
After you checked that all the premigration indexes with documents before $PREMIGRATION_END_DATE are migrated to OpenSearch, continue with the steps in Installing OpenSearch and migrating Elasticsearch data.