Migrating data protection rules (IBM Knowledge Catalog)
Use API calls or cpd-cli
commands to export and import data protection rules within a Cloud Pak for Data cluster or between different clusters.
You can export and import data protection rules with APIs. Alternatively, you can use the integrated Cloud Pak for Data command-line interface (cpd-cli
) to export data protection rules from any system and import the rules into the
same or different system. Links are maintained to glossary artifacts, catalogs, assets, and users. This task is also handy for backing up and restoring your data protection rules on the same or different system.
Choose either API or cpd-cli
commands:
Prerequisites
Required permission
To complete this task, you must be an administrator of the project (namespace) where you deployed the IBM Knowledge Catalog service.
Before you begin
-
All artifacts, users, user groups, user roles, and assets that are used in rules must be imported correctly to the target system before the exported rules are imported.
-
To avoid any errors and changes to the enforcement behavior, you must verify that the Manage rule settings between the source and target systems are configured the same.
-
Before you import a rule, verify whether you have custom predicates in the exported rules. If so, you must create these custom predicates in the target system before you can import.
Limitations
The unmasked original data can persist in temporary storage for the following scenarios:
- A cache of data to display in the Example data section when a data protection rule is created or edit.
- A masked copy of the data in Cloud Object Storage for rapid recall of the masked data.
- A profile data might not store the entire data but store an aggregation of the data, which can include values that are found in the data.
Migrating data protection rules by using API
Export
To export from one installation to another, set external_export
to true
by running the following API:
curl -k -X GET -H 'Content-Type: application/json' -H "Authorization:Bearer $token" '$url/v3/enforcement/rules/export?external_export=true'
An exported_file.json
file is generated that can be used for import.
Import
- Before you import between different installations, generate a report of all rules that succeed or fail in matching names to IDs in the target system. By running the following API that sets the
commit
parameter tofalse
:
curl -k -X POST -H 'Content-Type: application/octet-stream' -H "Authorization:Bearer $token" '$url/v3/enforcement/rules/import?external_import=true&commit=false' -d "@./exported_file.json"
Before you commit the rules, use the following report to identify any rules that fails to match IDs in the target system. When you run the import with external_import
set to true
, no rules are created and committed
when the commit
parameter is set to false
:
{
"failedList": [
{
"name": "export name4",
"index": 4,
"ruleGuidInRequest": "abcd123e-123b-1ab2-a123-ab123456c7d1",
"reason": "WDPPS9040E: Import rules failed to match export component ids: [id=123a4b56-1a23-1234-a1bc-12a3b45cd6ef_123a4b56-1a23-1a23-1a23-12ab345cd67e]"
}
],
"matchedList": [
{
"name": "export name1",
"index": 1,
"ruleGuidInRequest": "12abc456-1234-12a3-a1bc-a1abc2345abc"
},
{
"name": "export name2",
"index": 2,
"ruleGuidInRequest": "123456a7-1abc-1a2b-a1bc-ab1cd234ef56"
},
{
"name": "export name3",
"index": 3,
"ruleGuidInRequest": "a123bcd4-a1b2-1234-1234-a1234567b123"
}
]
}
- Run the following API to set the
commit
parameter totrue
, such that all or none of the rules can be committed. If one of the rules fails, the whole import fails:
curl -k -X POST -H 'Content-Type: application/octet-stream' -H "Authorization:Bearer $token" '$url/v3/enforcement/rules/import?external_import=true&commit=true' -d "@./exported_file.json"
A sample response when importing the export package (exported_file.json
file) is successful:
SUCCEEDED
{
"importedList": [
{
"name": "export name1",
"index": 1,
"guid": "a1b2cd3e-4567-8f90-gh12-345i6jklm78n",
"ruleGuidInRequest": "12abc345-6789-01d2-e3fg-h4ijk5678lmn"
},
{
"name": "export name2",
"index": 2,
"guid": "12a234b5-c678-90d1-ef2g-34567h8i9012",
"ruleGuidInRequest": "123456a7-8bcd-9e0f-g1hi-jk2lm345no67"
},
{
"name": "export name3",
"index": 3,
"guid": "1a234b5c-d6e7-89f0-1234-5g6hi7j8901k",
"ruleGuidInRequest": "aa123bcd4-e5f6-7890-1234-g5678901h234"
},
{
"name": "export name4",
"index": 4,
"guid": "12a345bc-6789-0d12-3ef4-g5h6ijk789lm",
"ruleGuidInRequest": "abcd123e-456f-7gh8-i901-jk234567l8m9"
}
]
}
A sample response of when importing the export package (exported_file.json
file) fails:
FAILED
{
"trace": "a1bc2d3e4fg5h6ijklmn7o9pq",
"errors": [
{
"code": "WDPPS9043E",
"message": "WDPPS9043E: Import rules failed: {\"failedList\":[{\"name\":\"export name4\",\"index\":4,\"ruleGuidInRequest\":\"abcd123e-123b-1ab2-a123-ab123456c7d1\",\"reason\":\"WDPPS9040E: Import rules failed to match export component ids: [id=123a4b56-7c89-0123-d4ef-56g7h89ij0kl_123m4n56-7o89-0p12-3q45-67rs890tu12v]\"}]}"
}
]
}
Migrating data protection rules by using cpd-cli
commands
Before you begin
- The
cpd-cli
command-line interface and related modules must be installed. See Installing the Cloud Pak for Data command-line interface (cpd-cli
). - Verify that you created a
cpd-cli
profile on your workstation. See Creating a profile to use the cpd-cli management commands. - After you obtain the API key, complete the profile setup by running the following commands:
cpd-cli config users set admin --username <username> --apikey <apikey>
cpd-cli config profiles set default --user admin --url <url>
Export
To export data protection rules from the IBM Knowledge Catalog service, complete the following steps:
- Prepare a YAML file with the export specification. Create an
export.yaml
file with the following parameters:
- parameter: EXPORT_HOST
value: The hostname of the cluster from which you want to export data. - parameter: EXTERNAL_EXPORT
value:true
orfalse
. This parameter is required. To export between different systems, set this parameter totrue
so that the IDs in rules from source system are converted to names at export. - parameter: AUTH_STRING
value: The access token for the source system. You can specify either a bearer orZenApiKey
token. For more information about generating aZenApiKey
token, see Generating an API authorization token. For more information about generating a bearer token, see Generating a bearer token.
Make sure the key and indentation are as in the following example:
policy-api-aux:
EXPORT_HOST: <hostname>
EXTERNAL_EXPORT: true
AUTH_STRING: Bearer <token> | ZenApiKey <token>
If the export.yaml
file is not provided, the following default values are used during export:
EXPORT_HOST: <hostname>
- current hostEXTERNAL_EXPORT: true
AUTH_STRING:
- The token is obtained by thecpd-cli
for the user defined in profiles specified by the required--profile
parameter.
- Go to the directory where the
cpd-cli
command-line interface is installed and run the export command:
cpd-cli export-import export create -c <component_name> --namespace <namespace> <export_name> -f <export_yaml_file> --log-level=debug --verbose --profile=default
Replace the following values:
- Replace component_name with the value
policy-api
. - Replace namespace with the namespace where the IBM Knowledge Catalog service is deployed.
- Replace export_name with the name of the export task.
- Replace export_yaml_file with the path to the YAML file with the export specification.
Example:
cpd-cli export-import export create -c policy-api --namespace wkc export1 -f /root/export.yaml --log-level=debug --verbose --profile=default
An exported .json
file is created at ../data/export.json
of the export package.
- Wait for export to complete. You can check the export status by running the following command:
cpd-cli export-import export status --namespace <namespace> <export_name> --profile=default
Replace the following values:
- Replace namespace with the namespace where the IBM Knowledge Catalog service is deployed.
- Replace export_name with the name of the export task.
Example of replaced values:
cpd-cli export-import export status --namespace wkc export1 --profile=default
Tip: A status of SUCCEEDED
indicates only the completion of the export job. If no data was exported despite the success message, check the log files. The export might have failed, for example, due to permission
errors.
- Download the export result:
cpd-cli export-import export download --namespace <namespace> <export_name> --profile=default
Replace the following values:
- Replace namespace with the namespace where the IBM Knowledge Catalog service is deployed.
- Replace export_name with the name of the export task.
Example of replaced values:
cpd-cli export-import export download --namespace wkc export1 --profile=default
The name of the export package has this format: cpd-exports-<export_name>-<timestamp>-data.tar
- (Optional) At any time, you can download the log file for a specific export job by using the following command:
cpd-cli export-import export logs --namespace <namespace> <export_name> --profile=default
Replace the following values:
- Replace namespace with the namespace where the IBM Knowledge Catalog service is deployed.
- Replace export_name with the name of the export task.
Example of replaced values:
cpd-cli export-import export logs --namespace wkc export1 --profile=default
Import
To import data protection rules to the IBM Knowledge Catalog service, complete the following steps. If you want to import from an export created on a different cluster, log in to the remote cluster before you follow the steps:
- Prepare a YAML file with the import specification. Create an
import.yaml
file with the following parameters:
- parameter: IMPORT_HOST
value: The hostname of the cluster from which you want to import data. - parameter: EXTERNAL_IMPORT
value:true
orfalse
. This parameter is required. To import between two different systems, set this parameter totrue
so that the names in rules are converted to IDs in the target system at import. - parameter: IMPORT_COMMIT
value:true
orfalse
. This parameter is required. To enable thisIMPORT_COMMIT
parameter, you must set theEXTERNAL_IMPORT
parameter totrue
. For external imports, set this parameter totrue
to commit rules at import. If one rule fails to commit, the whole import fails. To get a report of rules that succeed in matching names to IDs at import without committing the rules, set this parameter tofalse
. - parameter: AUTH_STRING
value: The access token for the target system. You can specify either a bearer orZenApiKey
token. For more information about generating aZenApiKey
token, see Generating an API authorization token. For more information about generating a bearer token, see Generating a bearer token.
Make sure the key and indentation are as in the following example:
policy-api-aux:
IMPORT_HOST: <hostname>
EXTERNAL_IMPORT: true
IMPORT_COMMIT: true
AUTH_STRING: Bearer <token> | ZenApiKey <token>
If the import.yaml
file is not provided, the following default values are used during import:
IMPORT_HOST: <hostname>
- current hostEXTERNAL_IMPORT: true
IMPORT_COMMIT: true
AUTH_STRING:
- The token is obtained by thecpd-cli
for the user defined in profiles specified by the required --profile parameter.
- Go to the directory where the
cpd-cli
command-line interface is installed. Upload the downloaded .tar file from the export package by running the following command:
cpd-cli export-import export upload --namespace <namespace> -f <file_data.tar> --profile=default
Replace the following values:
- Replace namespace with the namespace where the IBM Knowledge Catalog service is deployed.
- Replace file_data.tar with the name of your export package.
Example of replaced values:
cpd-cli export-import export upload --namespace wkc -f cpd-exports-export1-20231128142513-data.tar --profile=default
- (Optional) Run the following command to verify the upload:
cpd-cli export-import export list --namespace <namespace> --log-level=debug --verbose --profile=default
Replace namespace with the namespace where the IBM Knowledge Catalog service is deployed.
Example of replaced values:
cpd-cli export-import export list --namespace wkc --log-level=debug --verbose --profile=default
- Run the following command to import the data to the IBM Knowledge Catalog service:
cpd-cli export-import import create --from-export <export_name> --namespace <namespace> <import_name> --log-level=debug --verbose -f <import_yaml_file> --profile=default
Replace the following values:
- Replace export_name with the name of the export task.
- Replace namespace with the namespace where the IBM Knowledge Catalog service is deployed.
- Replace import_name with the name of the import task.
- Replace import_yaml_file with the path to the YAML file with the import specification.
Example of replaced values:
cpd-cli export-import import create --from-export export1 --namespace wkc import1 --log-level=debug --verbose -f /root/import.yaml --profile=default
- Wait for import to complete. You can check the import status by running the following command:
cpd-cli export-import import status --namespace <namespace> <import_name> --profile=default
Replace the following values:
- Replace namespace with the namespace where the IBM Knowledge Catalog service is deployed.
- Replace import_name with the name of the import task.
Example of replaced values:
cpd-cli export-import import status --namespace wkc import1 --profile=default
Tip: A status of SUCCEEDED
indicates only the completion of the import job. If no data was imported despite the success message, check the log files. The import might have failed, for example, due to permission
errors.
- (Optional) Download a log file by running the following commands:
cpd-cli export-import import logs --namespace <namespace> <import name> --profile=default
Replace the following values:
- Replace namespace with the namespace where the IBM Knowledge Catalog service is deployed.
- Replace import_name with the name of the import task.
Example of replaced values:
cpd-cli export-import import logs --namespace wkc import1 --profile=default
Learn more
Parent topic: Administering IBM Knowledge Catalog