Migrating catalog assets (Watson Knowledge Catalog)

Use cpd-cli commands to export and import catalog assets within a single Cloud Pak for Data cluster or between different clusters.

The migration of catalog assets is supported for these Cloud Pak for Data versions:

Export catalog assets from Cloud Pak for Data 3.0, 3.5, and 4.0 and later
Import catalog assets to Cloud Pak for Data 4.0 and later

The following catalog assets can be imported and exported:

connection
data_asset
data_definition
data_flow
data_intg_flow
data_intg_subflow
data_rule ( supported in IBM Cloud Pak for Data 4.6.3 and later)
data_rule_definition ( supported in IBM Cloud Pak for Data 4.6.3 and later)
do_decision_asset
environment
folder_asset
hardware_specification
job
metadata_import
notebook
package_extension
parameter_set
script
shiny_asset
software_specification
wml_experiment
wml_function
wml_model
wml_model_definition
wml_pipeline

Required permission: To complete this task, you must be an administrator of the project (namespace) where you deployed the Watson Knowledge Catalog service, and an administrator of the Red Hat OpenShift project.

Prerequisites

The cpd-cli command-line interface and related modules must be installed. For more information, see installing the product metadata backup and migration service.

The catalog-api-aux module must be registered. To verify that the module is available, run the following command:

cpd-cli list aux-modules

The module should be listed in the command output.

Exporting catalog assets

To export catalog assets from the Watson Knowledge Catalog service, complete the following tasks:

Determine the scope of the export. If you want to export all assets from all catalogs, projects and spaces, go to the next step. If you want to select which assets to export, create a YAML file with the export specification. Create an export.yaml file with the following structure:

catalog-api-aux:
  # exportspec specifies which assets to export
  exportspec: 'EXPORT_JSON_STRING'

Replace the EXPORT_JSON_STRING variable with a JSON string that specifies assets to export. For example:

catalog-api-aux:
  exportspec: '{"catalog": {    "container_specs": [{   "guids": ["08f248d3-0c5f-4193-a16d-e3c0cc7f7d94"], "all_assets": true}  ]} }'

The JSON string must be placed in one line and wrapped with single quote characters. The following examples show JSON strings added on multiple lines only for clarity purposes.

Export all assets from the list of specified catalogs, projects, or spaces:

'{
  "project": { <<---- this is the type of asset container: catalog, project, or space
    "container_specs": [
      {
        "guids": [
          "00cf5b102-17b3-4638-95df-309a2a443137",  <<---- these are the ids of asset containers
          "e14a187d-4fc8-4a5d-aee1-e2ad4ec60b29"
        ],
        "all_assets": true
      }
    ]
  },
  "catalog": { <<---- this is the type of asset container: catalog, project, or space
    "container_specs": [
      {
        "guids": [
          "ab3d18a4-015f-4bb1-ae13-4ee9faf0382a"  <<---- these are the ids of asset containers
        ],
        "all_assets": true
      }
    ]
  }
}'

Export specific assets from the list of specified catalogs, projects, or spaces:

'{
  "space": { <<---- this is the type of asset container: catalog, project, or space
    "container_specs": [
      {
        "guids": [
          "4b014869-fc0e-4919-bc93-3fa8c40bcd37"  <<---- this is the id of asset container
        ],
        "asset_specs": {
          "asset_ids": [
            "e14a187d-4fc8-4a5d-aee1-e2ad4ec60b29",  <<---- these are the ids of assets
            "e2395865-5503-49da-b708-cfbf2fb1b646"
          ]
        }
      }
    ]
  }
}'

Export assets of specific types in from the list of specified catalogs, projects, or spaces:

'{
  "space": { <<---- this is the type of asset container: catalog, project, or space
    "container_specs": [
      {
        "guids": [
          "4b014869-fc0e-4919-bc93-3fa8c40bcd37",  <<---- these are the ids of asset containers
          "866613f8-4102-4cd2-be6e-ee46ea6c468b"
        ],
        "asset_specs": {
          "asset_types": [
            "data_asset",  <<---- these are the types of assets
            "notebook"
          ]
        }
      }
    ]
  }
}'

Go to the directory where the cpd-cli command-line interface is installed and run the export command:

./cpd-cli -n Namespace export create Export_name \
  --component catalog-api \
  --values Override_yaml_file_location

Replace the following values:

Replace Namespace with the namespace where the Watson Knowledge Catalog service is deployed.
Replace Export_name with the name of the export task.
Replace Override_yaml_file_location with the path to the YAML file with the export specification. Do not include this variable if you want to export all catalog assets.

Example:

./cpd-cli -n Namespace export create export-test --component catalog-api --values /root/export.yaml

Wait for export to complete. You can check the export status by using this command:
```
./cpd-cli -n Namespace export status Export_name
```
Replace values as described in step 2.

Example:
```
./cpd-cli -n Namespace export status export-test
```
Download the export result:
```
./cpd-cli -n Namespace export download Export_Name
```
Replace values as described in step 2.

Example:
```
./cpd-cli -n Namespace export download export-test
```
The name of the exported package has this format: cpd-exports-<export_name>-<timestamp>-data.tar

Importing catalog assets

To import catalog assets to the Watson Knowledge Catalog service, complete the following tasks. If you want to import from an export created on a different cluster, copy exported archive to the current cluster, and log in to the current cluster.

Go to the directory where the cpd-cli command-line interface is installed and upload the exported package:

./cpd-cli -n Namespace export upload \
-f Export_archive_file_name

Replace the following values:

Replace Namespace with the namespace where the Watson Knowledge Catalog service is deployed.
Replace Export_archive_file_name with the location of the exported package.

Example:

./cpd-cli -n Namespace export upload \
-f /root/cpd-exports-export-test-20211025081052-data.tar

Run the following command to import the data to the Watson Knowledge Catalog service:

./cpd-cli -n Namespace import create Import_name \
 --from-export Export_name

Replace the following values:

Replace Namespace with the namespace where the Watson Knowledge Catalog service is deployed.
Replace Import_name with the name of the import task.
Replace Export_name with the name of the export task.

Example:

./cpd-cli -n Namespace import create import-test \
--from-export export-test

Wait for import to complete. You can check the import status by using this command:
```
./cpd-cli -n Namespace import status Import_name
```
Replace values as described in step 2.

Example:
```
 ./cpd-cli -n Namespace import status import-test
```

Important: After you complete importing the catalog assets, confirm whether two IADB connections existed in IMAM. If they did, you must delete the old IADB connection, otherwise, it'll cause an auto discovery or quickscan failure. For example, an old IADB connection might be jdbc:db2/is:xmeterdocker:50000/IADB and must be removed, while you keep jdbc:db2/c-db2oltp-iis-db2u:50000.

Learn more

Installing the product metadata backup and migration service

Parent topic: Administering Watson Knowledge Catalog