Backup/Export and Restore Transformation Advisor Data

Backup/Export of Transformation Advisor data

Option 1: Exporting using Transformation Advisor API

For Transformation Advisor on Red Hat OpenShift, using the Transformation Advisor API is the best option for backing up the data. For more information, see Exporting and importing Transformation Advisor data via HTTP endpoints document. In particular, see the section entitled Export all zip files for a workspace that describes the bulk export API.

Option 2: Save data directories

Save data directories for Transformation Advisor Local

For a Transformation Advisor Local install, saving the data directories is the best option for backing up the data. You may also use the Transformation Advisor API to back up the data for a Local installation, but saving the data directory is likely more convenient. Go to the location where Transformation Advisor Local is installed. Locate the data and graph_data directories and copy them to your desired backup location. You also need to copy the .neo4j_pass file.

cp -a <some location>/data <backup location>/data
cp -a <some location>/graph_data <backup location>/graph_data
cp -a <some location>/scripts/.neo4j_pass <backup location>/scripts/

Save data directories for Transformation Advisor on Red Hat OpenShift

It is not recommended to directly back up or restore data directories for Transformation Advisor on Red Hat OpenShift.

Restore/Import of Transformation Advisor data

Importing using Transformation Advisor API

If you have previously backed up your data using the Transformation Advisor API, you should use the API to import or restore that data to Transformation Advisor. See Exporting and importing Transformation Advisor data via HTTP endpoints for more details on this option.

Restoring data directories

Restore data directories for Transformation Advisor Local

1. cd to the location of Transformation Advisor Local (the exploded Transformation Advisor downloaded zip)
2. Restore the data to the new location:
   • cp -a <backup location>/data .
   • cp -a <backup location>/graph_data .
   • cp -a <backup location>/scripts/.neo4j_pass scripts/
3. In the new location install Transformation Advisor:
   • ./launch.sh
   • Accept the license agreement.
   • Choose Option 1 (Install).

Restore data directories for Transformation Advisor on Red Hat OpenShift

It is not recommended to directly back up or restore data directories for Transformation Advisor on Red Hat OpenShift.

Rolling back to a previous version of Transformation Advisor

When rolling back to a previous version of Transformation Advisor, it is important to use data that is compatible with that version. Transformation Advisor does not automatically convert newer data for older versions. For example:

1. Install Transformation Advisor 3.0.0
2. Back up data for 3.0.0
3. Upgrade to 3.1.0
4. Back up data for 3.1.0
5. Roll back to 3.0.0

In this scenario, you must use the data originally backed up from Transformation Advisor 3.0.0. You cannot use the backup from 3.1.0.

Advanced Backup and Restore Options

The following advanced options should only be used if you are unsuccessful with other backup and restore options.

Preserving the PersistentVolumeClaim

You can perform an uninstall of Transformation Advisor but also preserve the PersistentVolumeClaim that it uses. This procedure is useful in the following scenarios:

• If you want to uninstall a version of Transformation Advisor and install a new version, but want an easy way to revert to the old version. This is recommended when moving from 2.5.x to 3.0.x.
• If you want to keep your data across Transformation Advisor version upgrades when an automatic seamless upgrade is not possible (for example, when upgrading to 2.4.x from 2.3.x). This approach will not preserve data when going from 2.5.x to 3.0.x.

The limitation with this approach is that you can only reuse the PersistentVolumeClaim in the same namespace it was originally used.

kubectl get pvc <my-pvc-name> -n <ta-namespace> -o yaml | grep ownerReferences

If that command shows no ownerReferences, then you do not need this procedure; you can delete Transformation Advisor and your PersistentVolumeClaim will remain. Always back up data beforehand.

To preserve the PersistentVolumeClaim, follow these steps:

1. Remove the Transformation Advisor subscription and clusterserviceversion. This also removes the operator. Replace <ta-namespace> with the actual namespace where the install exists.

   export subscription=$(kubectl get subscription --no-headers -n <ta-namespace> | awk '{print $1}')
   kubectl delete subscription ${subscription} -n <ta-namespace> >/dev/null 2>&1
   
   export csv=$(kubectl get csv --no-headers -n <ta-namespace> | awk '{print $1}')
   kubectl delete csv ${csv} -n <ta-namespace>
   

   Wait until the operator pod disappears. Check with oc get pods until ta-operator-XXX is gone.

2. Detach the PersistentVolumeClaim from the Transformation Advisor instance to allow deletion while keeping the claim.

   export pvc_name=$(oc get deployment -n <ta-namespace> | grep couchdb | awk '{print$1}')
   kubectl patch pvc ${pvc_name} -n <ta-namespace> --type=json -p='[{"op": "remove", "path": "/metadata/ownerReferences"}]'
   

3. Remove the Transformation Advisor instance.

   # Run this command in the background using '&'
   kubectl delete transadvs.charts.ta.cloud.ibm.com/ta -n <ta-namespace> &
   kubectl patch transadvs.charts.ta.cloud.ibm.com/ta -p '{"metadata":{"finalizers":[]}}' --type=merge -n <ta-namespace>
   

   Wait until all pods in the namespace are gone.

4. Remove the CustomResourceDefinition.

   # Run this command in the background using '&'
   kubectl delete crd/transadvs.charts.ta.cloud.ibm.com &
   kubectl patch crd/transadvs.charts.ta.cloud.ibm.com -n <ta-namespace> -p '{"metadata":{"finalizers":[]}}' --type=merge
   

5. Clean up other Transformation Advisor resources in the namespace.

   kubectl delete secret transformation-advisor-secret -n <ta-namespace>
   
   export operator_group=$(oc get operatorgroup -n <ta-namespace> --no-headers | awk '{print $1}')
   kubectl delete operatorgroup ${operator_group} -n <ta-namespace>
   
   kubectl delete clusterrolebinding <ta-namespace>-cluster-admin -n <ta-namespace>;
   

6. When reusing the preserved PersistentVolumeClaim, see Configure storage and Installing for configuration details.

   • If installing via the OpenShift UI, ensure persistence properties match the preserved claim (for example, same accessMode).

     ...
       persistence:
         enabled: true
         accessMode: "ReadWriteOnce"
         size: 8Gi
         useDynamicProvisioning: true
         existingClaim: "<my-ta-pvc>"
         storageClassName: ""
         supplementalGroups: []
     ...
     

   • If installing with the CASE installer, specify the existing claim using --persistenceClaim <my-ta-pvc>.

7. Optional: After installing the new version of Transformation Advisor, you may attach the PersistentVolumeClaim to the new instance. This restores default behavior, where deleting the instance also deletes its claim.

   export uid=$(oc get transadvs.ta.ibm.com/ta -o yaml -n <ta-namespace> | grep uid | awk 'NR==1{print$2}')
   
   kubectl patch pvc <my-ta-pvc> --type=json -p='[{"op": "add", "path": "/metadata/ownerReferences", "value": [{"apiVersion": "ta.ibm.com/v1", "blockOwnerDeletion": true, "controller": true, "kind": "TransAdv", "name": "ta", "uid": "'${uid}'"}]}]'
   

Recovery Point Objective

The most important Transformation Advisor data are the archives generated during a Transformation Advisor data collection. These archives should be backed up and retained per your organization’s policy.

The only data that cannot be restored from the original data collection archives are:

• Manually created application groups
• Modifications to collection names

Once data has been loaded into the application, you can retrieve and back it up using the API. The bulkExport API returns an archive of all Transformation Advisor data (including modifications made to groups, workspaces, and collections). That archive can be imported to a new installation using the bulkImport API.

Periodic backups using the bulkExport API are recommended. Alternatively, or in addition, you may also periodically back up the database (see Back-up Options).

Consider the following factors when deciding the Recovery Point Objective:

• Frequency of new data uploads to Transformation Advisor
• Frequency and volume of modifications to uploaded data (creating, editing, or deleting groups; editing workspace or collection names)
• Availability of backed-up original Transformation Advisor data collection archives