Known issues and limitations for IBM Knowledge Catalog
The following known issues and limitations apply to IBM Knowledge Catalog.
Known issues
General
- Assets imported with the user
admin
instead ofcpadmin
- Heavy I/O load can cause out-of-memory failures of the wkc-db2u instance
- Search bar returning incorrect results
- Global asset types when created as
decorator
show up in the UI - Assessment link in email notifications doesn't work
Installing, upgrading, and uninstalling
- In multi-pod configurations, event handling errors can occur
- Upgrades fail when the common core services cannot be upgraded
- When uninstalling Manta Data Lineage, re-installing IBM Knowledge Catalog runs into issues
- Pods with dependency on common core services fail when the common core services are uninstalled and re-installed
- After the upgrade to 5.1, predefined roles are missing permissions
- During the upgrade from 5.0.3 to 5.1.0, the
wkc-data-rules
pod keeps restarting - Installing IBM Knowledge Catalog fails because of the Analytics Engine
- After upgrading to 5.1.0, or later, the ingestion service pod service crashes
- IBM Knowledge Catalog portal-catalog pod out of sync after upgrade
Migration and removal of legacy functions
For known issues with migration from InfoSphere Information Server, see Known issues for migration from InfoSphere Information Server.
Catalogs and Projects
- Previewing assets with a PDF extension
- Details for masked columns display incorrectly
- Unauthorized users might have access to profiling results
- Cannot run import operations on a container package exported from another Cloud Pak for Data cluster
- Data protection rules do not apply to column names that contain spaces
- Preview of data from file-based connections other than IBM Cloud Object Storage is not fully supported
- Scroll bar is not visible when adding assets to a project on MacOS
- Unexpected assets filtering results in catalogs- The asset metadata export CSV file includes unexpected asset properties
- Can't create a connection if you're including a reference connection
- Migrating data source definitions from the Platform assets catalog will fail
- Can't profile tables with clob and blob data types
- Migrated connections aren't listed during importing from the asset browser
Governance artifacts
- Error
Couldn't fetch reference data values
shows up on screen after publishing reference data - Publishing large reference data sets fails with Db2 transaction log full
- Imported data assets with assigned out-of-the-box data classes or terms have incorrect identifiers resulting in no enforcement of data protection rules
- Business terms remain after the semantic automation layer integration is deleted from IBM watsonx.data
- Importing artifacts from a zip file fails but reports success when category is created manually
- A custom property is still visible after restricting it to categories
Metadata import
- Business data lineage is incomplete for the metadata imports with Get ETL job lineage or Get BI report lineage goals
- Assets are not imported from the IBM Cognos Analytics source when the content language is set to Japanese
- When you import a project from a .zip file, the metadata import asset is not imported
- Lineage metadata cannot be imported from the Informatica PowerCenter connection
- Dummy assets get created for any file assets that come from Amazon S3 to show the complete business data lineage if Get ETL job lineage is performed
- Related assets are not displayed in lineage that was created from the Greenplum connection
- Cannot delete added inputs from a file
- Metadata import jobs might fail for any Generic JDBC connection that uses a vaulted JDBC URL
- SocketTimeoutException during metadata import
Metadata enrichment
- Running primary key or relations analysis doesn't update the enrichment and review statuses
- Writing metadata enrichment output to an earlier version of Apache Hive than 3.0.0
- Issues with the Microsoft Excel add-in
- Republishing doesn't update primary key information in catalog
- Publishing SQL-based data assets from the metadata enrichment results isn't blocked
- Updates of business term abbreviations are not considered in name generation
- Masked data might be profiled when the data source is IBM watsonx.data
Data quality
- Rules with multiple joins might return incorrect results for data assets from Apache Cassandra, Apache Hive, MongoDB, or Oracle data sources
- Rules bound to columns of the data type NUMERIC in data assets from Oracle data sources might not work
- Runs of migrated data quality rules complete with warnings
- Data quality output settings show some unsupported parameters
MANTA Automated Data Lineage for IBM Cloud Pak for Data
-
Metadata import jobs for getting lineage might take very long to complete
-
Columns are displayed as numbers for a DataStage job lineage in the catalog
-
MANTA Automated Data Lineage will not function properly on IBM Knowledge Catalog Standard
-
Not all stages are displayed in technical data lineage graph for the imported DataStage ETL flow
-
Can't 'Get lineage' with a DB2 connection in FIPS environments
Business lineage
- Lineage metadata don’t show on Knowledge Graph after upgrading
- Business data lineage is incomplete for the metadata imports with Get ETL job lineage or Get BI report lineage goals
- Hops for components and columns of components work only inside the data integration flow area of an expanded job node
- Cannot import lineage mappings
Relationship explorer
- Relationship explorer is not working after installing the IBM Software Hub Version 5.1
- Error Your flow is empty shows up on screen after clicking Explore relationships in the catalog
- Cannot view project metadata import assets in the relationship explorer
Reporting
Also see:
Limitations
Catalogs and Projects
- Default catalog is missing
- Special or double-byte characters in the data asset name are truncated on download
- Catalog UI does not update when changes are made to the asset metadata
- A blank page might be rendered when you search for terms while manually assigning terms to a catalog asset
- Profiling in catalogs, projects, and metadata enrichment might fail for Teradata connections
- Catalog asset search doesn't support special characters
- Can't add individual group members as asset Members
- Long names of the asset owners get truncated when hovering over their avatars
- Duplicate actions fail if dynamic IP address are used
- Project assets that are added while you create segmented data assets might not be available for selection
- An extra path to manage catalogs in the navigation menu
Governance artifacts
- Cannot use CSV to move data class between Cloud Pak for Data instances
- Unable to use masked data in visualizations from data assets imported from version 4.8 or earlier
Metadata import
- Metadata import jobs might be stuck due to issues related to RabbitMQ
- Data assets might not be imported when running an ETL job lineage import for DataStage flows
- When lineage import fails, no details in the job log are available
- When a job for importing lineage metadata hangs, it cannot be stopped
- Only files with .sql extension can be provided as manual input for metadata import from the Oracle and PostgreSQL sources
Metadata enrichment
- In some cases, you might not see the full log of a metadata enrichment job run in the UI
- Schema information might be missing when you filter enrichment results
- Profiling in catalogs, projects, and metadata enrichment might fail for Teradata connections
- For assets from SAP OData sources, the metadata enrichment results do not show the table type
Data quality
Business lineage
General issues
You might encounter these known issues and restrictions when you work with the IBM Knowledge Catalog service.
Assets imported with the user admin
instead of cpadmin
For Cloud Pak for Data clusters with Identity Management Service enabled, the default administrator is cpadmin
. However, for import, the default administrative user admin
is used. Therefore, the assets are imported
with the admin
user instead of cpadmin
.
Applies to: 5.0.0 and later
Workaround:
Before running the import, apply the following workaround:
-
Edit the config map by executing
oc edit cm catalog-api-exim-cm
-
Manually update the environment variable
admin_username
inimport-job.spec.template.spec.env
from:- name: admin_username value: ${admin_username}
to:
- name: admin_username value: cpadmin
Heavy I/O load can cause out-of-memory failures of the wkc-db2u instance
Applies to: 5.0.0 and later
After a metadata enrichment job fails, you see that the pods for the glossary service, data quality rules, and wkc-db2 were restarted. When you check the status of the wkc-db2 pod, you see the following error:
Error:
terminated:
exitCode: 143
reason: OOMKilled
This error indicates that resource limits must be increased.
Workaround: Scale up the Db2 instance for the IBM Knowledge Catalog service on Cloud Pak for Data to enhance high availability and increase processing capacity for the IBM Knowledge Catalog service. Allocate additional memory and CPU resources to the existing Db2 deployment by completing these steps:
-
Specify the CPU and memory limit. In this example, CPU is set to 8 vCPU and memory is set to 15 Gi. Modify the values according to your needs.
oc patch db2ucluster db2oltp-wkc --type=merge --patch '{"spec": { "podConfig": { "db2u": { "resource": { "db2u": { "limits": { "cpu": "8", "memory": "15Gi" } } } } } }}'
-
Wait for the
c-db2oltp-wkc-db2u-0
pod to restart.
For more information, see Scaling up Db2 for IBM Knowledge Catalog. If needed, also complete steps 3 to 6 of the described procedure.
Search bar returning incorrect results
-
Searching for assets when using the search bar returns unexpected results if only one or two characters are used.
Applies to: 5.1 and later
Workaround: Type at least three characters in the search bar.
Global asset types when created as decorator
show up in the UI
Applies to: 5.1.0
Fixed in: 5.1.1
When working in the Catalog UI, data assets in every tenant have additional fields appearing on the UI. These are fields you do not have to fill out as the global asset types were marked as decorator
.
This only happens when
Watson Machine Learning is installed.
Workaround: To fix this issue, run the following as a cluster or namespace administrator and then patch synthetic_data
using the following steps:
- Get the
CPD_URL
:CPD_URL=$(oc get route -n <cpd instance namespace> cpd -o jsonpath="{.spec.host}")
- Get the
service-id-credentials
:SERVICE_ID_CRED=$(oc get secret -n <cpd instance namespace> wdp-service-id -o jsonpath="{.data.service-id-credentials}"| base64 -d)
- With the credentials, run the
PUT
command:curl -k -X 'PUT' \ "https://$CPD_URL/v2/asset_types/wx_synthetic_data" \ -H 'accept: application/json' \ -H "Authorization: Basic $SERVICE_ID_CRED" \ -H 'Content-Type: application/json' \ -d '{ "description": "watsonx.ai Synthetic Data asset type", "fields": [ { "key": "instructlab_synthetic_data_id", "type": "string", "facet": false, "is_array": false, "search_path": "instructlab_synthetic_data_id", "is_searchable_across_types": false } ], "relationships": [], "properties": { "custom": { "type": "object", "description": "caller defined properties", "placeholder": "Put a value here", "is_array": false, "required": false, "hidden": false, "readonly": false, "validated": false, "displayInChild": false, "restricted": false, "properties": { "{user-defined-property}": { "description": "custom property", "placeholder": "Put a value here", "is_array": false, "required": false, "hidden": false, "readonly": false, "validated": false, "displayInChild": false, "restricted": false } } }, "instructlab_synthetic_data_id": { "type": "string", "description": "InstructLab synthetic data id", "placeholder": "Put a value here", "is_array": false, "required": false, "hidden": false, "readonly": false, "validated": false, "displayInChild": false, "restricted": false, "label": { "default": "InstructLab Synthetic Data id" } }, "ml_version": { "type": "string", "description": "Internal version of the ml asset type", "placeholder": "Put a value here", "is_array": false, "required": true, "hidden": true, "readonly": true, "validated": false, "displayInChild": false, "restricted": false, "default_value": "1.0.1" }, "status": { "type": "string", "description": "Status for the asset readiness", "placeholder": "Put a value here", "is_array": false, "required": true, "hidden": false, "readonly": false, "validated": false, "displayInChild": false, "restricted": false, "values": [ "pending", "ready", "failed" ], "label": { "default": "Status" } } }, "localized_metadata_attributes": { "name": { "default": "watsonx.ai Synthetic Data" } }, "global_search_searchable": [], "attribute_only": false, "is_column_custom_attribute": false, "can_have_image": false, "allow_decorators": true }'
Assessment link in email notifications doesn't work
Applies to: 5.1.1 and later
In email notifications for data quality SLA assessment, the link to the assessment does not work due to an extra dot in the URL.
Workaround: To access the assessment, copy the link to a new browser window, remove the extra dot between ibm
and com
, and press Enter.
Installing, upgrading and uninstalling
You might encounter these known issues while installing, upgrading or uninstalling IBM Knowledge Catalog.
In multi-pod configurations, event handling errors can occur
Applies to: 5.1
In multi-pod configurations for semantic automation, map concepts and expand names may not work properly due to an error with event handling. This may result in failing Metadata enrichment jobs.
Workaround: To fix the error, you must reduce the number of replicas being used:
- Set the IBM Knowledge Catalog Standard operator to maintenance mode without changing the scale:
oc patch ikcstandard ikc-standard-cr --patch '{"spec": {"ignoreForMaintenance": true}}' --type='merge'
- Reduce the number of replicas:
oc scale deploy semantic-automation --replicas=1
Upgrades fail when the common core services cannot be upgraded
Applies to:
- Upgrades from Version 4.8 to 5.1.0
- Upgrades from Version 4.8 to 5.1.1
When you upgrade IBM Knowledge Catalog, the common core services upgrade fails with the following error:
'"Job" "projects-ui-refresh-users": Timed out waiting on resource'
This error occurs because the wkc_reporting_administrator
role is created without a name.
For more information on this issue, see Upgrades fail or are stuck in the InProgres state when common core services cannot be upgraded in the IBM Software Hub documentation.
When uninstalling Manta Data Lineage, re-installing IBM Knowledge Catalog runs into issues
Applies to: 5.1
You can install Manta Data Lineage with IBM Knowledge Catalog. If you uninstall Manta Data Lineage, and then try to re-install the wkc-cr
for IBM Knowledge Catalog, you may run into issues. The wkc-post-install-init
pod may fail to restart.
Workaround: To fix this issue, restart the ibm-nginx
pods, then restart the wkc-operator
pod. This will put the wkc-operator
in the completed state.
Pods with dependency on common core services fail when the common core services are uninstalled and re-installed
Applies to: 5.1.0
Fixed in: 5.1.1
Problems may occur when you re-install a service with a dependency on the common core services after you uninstall the service by running the cpd-cli manage delete-cr
command with the --include_dependency=true
option.
Workaround:
- Run the following command to check for pods in the
Error
state:oc get pod -n${PROJECT_CPD_INST_OPERANDS} |grep -v Completed |grep -v 1/1 |grep -v 2/2 |grep -v 3/3 |grep -v 4/4
- Get the error log for one of the pods in the
Error
state:oc logs -f <pod-name>
- Look for errors in the pods. The errors may show:
* Connection #0 to host internal-nginx-svc.wkc.svc.cluster.local left intact Status code from call to create global asset type: 504
- To delete the pods run:
oc delete pods -n ${PROJECT_CPD_INST_OPERANDS} -l component=<pod-name>
After the upgrade to 5.1, predefined roles are missing permissions
Applies to: 5.1.x
After the upgrade from IBM Knowledge Catalog 4.7.x or 4.8.x to IBM Knowledge Catalog 5.1.x or IBM Knowledge Catalog Premium 5.1.x, some permissions are missing from Data Engineer, Data Quality Analyst, and Data Steward roles. Users with these roles might not be able to run metadata imports or access any governance artifacts.
Workaround: To add any missing permissions to the Data Engineer, Data Quality Analyst, and Data Steward roles, restart the zen-watcher
pod by running the following
command:
oc delete pod $(oc get pod -n ${PROJECT_CPD_INST_OPERANDS} -o custom-columns="Name:metadata.name" -l app.kubernetes.io/component=zen-watcher --no-headers) -n ${PROJECT_CPD_INST_OPERANDS}
During the upgrade from 5.0.3 to 5.1.0, the wkc-data-rules
pod keeps restarting
Applies to: Upgrades from 5.0.3 to 5.1.0
When you upgrade IBM Knowledge Catalog running on Red Hat OpenShift on IBM Cloud from version 5.0.3 to version 5.1.0, migration of the DATAQUALITY schema might get stuck, which causes the wkc-data-rules
pod to restart continuously.
The pod log shows messages similar to the following ones:
Successfully validated 22 migrations (execution time 00:00.080s)
Current version of schema "DATAQUALITY": 4.6.18
Migrating schema "DATAQUALITY" to version 4.6.19 - Add More Indexes db2
Workaround: Run the migration script manually.
-
Identify the stuck
wkc-data-rules
pods. Get the pod information by running the following command:oc get pods | grep wdp-data-rules
Pods that are stuck in starting state show
0/1
as theirReady
status. Pods that are up and running show1/1
. -
From the stuck pod, copy all SQL files in the
/config/flyway/Flyway/flyway-6.2.4/sql
folder to the local file system:oc cp <stuckPodName>:/config/flyway/Flyway/flyway-6.2.4/sql /tmp/sql
Example:
oc rsh wkc-data-rules-554b547bbc-j652m:/config/flyway/Flyway/flyway-6.2.4/sql /tmp/sql
-
Copy all SQL files from the local temporary directory to the
/config/flyway/Flyway/flyway-6.2.4/sql
folder of the running pod.oc cp /tmp/sql <runningPodName>:/config/flyway/Flyway/flyway-6.2.4/
Example: oc cp /tmp/sql wkc-data-rules-76d9b7c666-6t4qr:/config/flyway/Flyway/flyway-6.2.4/
-
Log in to the running pod by using the following command:
oc rsh <runningPodName>
Example:
oc rsh wkc-data-rules-859f6f4c96-kdrb2
-
Run the migration script:
/config/flyway/Flyway/flyway-6.2.4/flyway migrate
Wait for the migration script to finish. Depending on how much data needs to be migrated, the script can take a while to complete.
Installing IBM Knowledge Catalog fails because of the Analytics Engine
Applies to: 5.1.0
Fixed in: 5.1.1
When installing IBM Knowledge Catalog, the installation may fail because the Analytics Engine spark-hb-java-trust-store
secret is missing.
Workaround: Run the following commands to fix this issue:
-
Get the
spark-hb-create-trust-store
pod name:oc get pods | grep spark-hb-create-trust-store
-
Restart the spark
truststore
pod. -
Wait for the pod to get into
Running
state. -
Force reconcile the Analytics Engine operator.
After upgrading to 5.1.0, or later, the ingestion service pod service crashes
Applies to: 5.1.0 and later
After upgrading IBM Knowledge Catalog to Version 5.1.0, or later, the Knowledge Graph ingestion service pod wdp-kg-ingestion-service-xxx
may crash.
Workaround: Run the following steps after upgrading:
-
Find the number of pods running ingeston service:
oc get deployment wdp-kg-ingestion-service -n ${PROJECT_CPD_INSTANCE}
-
Scale the ingestion service to 0:
oc scale deployment wdp-kg-ingestion-service --replicas=0 -n ${PROJECT_CPD_INSTANCE}
-
Wait for the ingestion service pods to end. Run the following command to check:
oc get pod -n ${PROJECT_CPD_INSTANCE} | grep ingestion
-
Get the credentials to login go to the RabbitMQ web console:
oc get secret rabbitmq-ha -o json -n ${PROJECT_CPD_INSTANCE}
Keep note of the values for the following:
rabbitmq-username
rabbitmq-password
Decode the username and password and use the decoded value when logging in to the RabbitMQ console:
echo <rabbitmq-password> | base64 -d
-
On your local machine run the following:
oc port-forward rabbitmq-ha-0 15671:15671
-
Open the following URL in your browser:
https://localhost:15671/#/queues
-
Using the RabbitMQ web console, find and delete all the queues starting with
kg*
. The complete list includes:kg-queue kg-queue-cams-bulk kg-queue-glossary kg-queue-policy
-
Scale the ingestion service back to the original size, which will depend on the installation of the chosen cluster. For example, you can run:
oc scale deployment wdp-kg-ingestion-service --replicas=1 -n ${PROJECT_CPD_INSTANCE}
-
Wait for all pods to become ready. Run the following to check the progress:
oc get pods -n ${PROJECT_CPD_INSTANCE} | grep ingestion
-
Using the RabbitMQ web console verify that the
kg*
queues are re-created. -
If you see assets that do not display the lineage graph correctly, run the re-sync operation:
oc create job -n ${PROJECT_CPD_INSTANCE} --from=cronjob/wkc-search-lineage-cronjob lineage-job
This operation is time-consuming and should only be run if necessary. For more information about re-sync, see Resync of lineage metadata.
IBM Knowledge Catalog portal-catalog pod out of sync after upgrade
Applies to: 5.1.1
When upgrading IBM Knowledge Catalog and changing editions, the portal-catalog
pod may become out of sync, leading to missing functionality that should be enabled from the upgrade.
Workaround: To enable the missing functionality, restart the portal-catalog
pod after upgrading IBM Knowledge Catalog.
Catalog and project issues
You might encounter these known issues and restrictions when you use catalogs.
Previewing assets with a PDF extension
Applies to: 5.1.0
Fixed in: 5.1.1
In a default Git project, when you try to preview an asset with the PDF extension, the PDF is downloaded instead. Preview the PDF by using a locally installed PDF viewer.
Details for masked columns display incorrectly
Applies to: 5.1.0 and later
In the asset preview page, which is known to happen for virtualized join views and watsonx.data connected data, the value for the Masked columns displays an incorrect count. In addition, the masked indicator icon image
is either missing or incorrectly displayed from the header of columns with masked data.
When a deep enforcement solution is configured to protect a data source, protection is subject to that configured deep enforcement solution to apply column masking. Each protection solution has its own semantics for applying data masking and thus masking indicators that are displayed in the user interface might not align with the actual columns masked.
For details on how masking rules apply to virtualized views, see Authorization model for views in the Cloud Pak for Data documentation.
Workaround: None.
Unauthorized users might have access to profiling results
Applies to: 5.1.0 and later
Users who are collaborators with any role in a project or a catalog can view an asset profile even if they don't have access to that asset at the data source level or in Data Virtualization.
Workaround: Before you add users as collaborators to a project or a catalog, make sure they are authorized to access the assets in the container and thus to view the asset profiles.
Cannot run import operations on a container package exported from another Cloud Pak for Data cluster
Applies to: 5.1.0 and later
When importing a container package exported from another Cloud Pak for Data cluster, permissions must be configured on the archive to allow export operations on the target cluster to access the files within the archive.
Workaround: To extract the export archive and modify permissions, complete the following steps:
- Create a temporary directory by running:
mkdir temp_directory
- Extract the archive by running:
tar -xvf cpd-exports-<export_name>-<timestamp>-data.tar --directory temp_directory
- Clients will need to run the following command on the target cluster:
Example output:oc get ns $CLUSTER_CPD_NAMESPACE -o=jsonpath='{@.metadata.annotations.openshift\.io/sa\.scc\.supplemental-groups}'
1000700000/10000
. - The first part of the output of the previous step (ex.
1000700000
) will need be applied as the new ownership on all files within the archive. Example:cd temp_directory/ chown -R 1000700000:1000700000 <export_name>
- Archive the fixed files with the directory, using the same export name and timestamp as the original exported tar:
tar -cvf cpd-exports-<export_name>-<timestamp>-data.tar <export_name>/
- Upload the archive.
Data protection rules don't apply to column names that contain spaces
Applies to: 5.1.0 and later
If a column name contains trailing or leading spaces during import, the column cannot be masked using data protection rules.
Workaround: When you're importing columns, ensure that column names don't contain trailing or leading spaces.
Preview of data from file-based connections other than IBM Cloud Object Storage is not fully supported
Applies to: 5.1.0 and later
Connected assets from file-based connections other than IBM Cloud Object Storage do not preview correctly. Data might appear in a table with missing and/or incorrect data. There is no workaround at this time.
Scroll bar is not visible when adding assets to a project on MacOS
When adding assets to a project, the scroll bar might not be available in the Selected assets table, showing a maximum of 5 assets.
Applies to: 5.1
Workaround: Change the MacOS settings:
- Click the Apple symbol in the top-left corner of your Mac's menu bar, then click System Settings.
- Scroll down and select Appearance.
- Under the Show scroll bars option, click the radio button next to Always.
Unexpected assets filtering results in catalogs
Applies to: 5.1.0 and later
In catalogs, when you are searching for an asset by using Find assets field, the search might return assets whose names don't match the name string that you typed in the search field and assets that contain a keyword in a property or a related item associated with the typed name string.
The asset metadata export CSV file includes unexpected asset properties
Applies to: 5.1.0 and later
Unexpected extra properties are included in the exported CSV file when you're adding and updating assets and asset metadata from CSV files to catalogs. Do not edit these properties.
Asset Property: data_asset>>dataset
Asset Property: data_asset>>mime_type
Asset Property: data_asset>>properties>>name
Asset Property: data_asset>>properties>>name
Asset Property: data_asset>>semantic_description>>confidence
Asset Property: data_asset>>semantic_description>>generated_date
Asset Property: data_asset>>semantic_description>>generated_description
Asset Property: data_asset>>semantic_description>>status
Asset Property: discovered_asset>>connection_id
Asset Property: discovered_asset>>connection_path
Asset Property: discovered_asset>>discovery_id
Asset Property: discovered_asset>>extended_metadata>>name
Asset Property: discovered_asset>>extended_metadata>>name
Asset Property: discovered_asset>>extended_metadata>>name
Asset Property: discovered_asset>>first_imported_timestamp
Asset Property: discovered_asset>>last_discovered_timestamp
Asset Property: discovered_asset>>last_imported_timestamp
Asset Property: discovered_asset>>last_job_action
Asset Property: discovered_asset>>last_job_run_id
Asset Property: discovered_asset>>metadata_import_id
Asset Property: discovered_asset>>outdated_reason
Asset Property: discovered_asset>>outdated_timestamp
Asset Property: key_analyses>>fk_assigned
Asset Property: key_analyses>>fk_assigned_as_pk
Asset Property: key_analyses>>fk_defined
Asset Property: key_analyses>>fk_defined_as_pk
Asset Property: key_analyses>>fk_suggested
Asset Property: key_analyses>>fk_suggested_as_pk
Asset Property: key_analyses>>key_analysis_area_id
Asset Property: key_analyses>>overlap_assigned
Asset Property: key_analyses>>overlap_suggested
Asset Property: key_analyses>>pk_assigned
Asset Property: key_analyses>>pk_defined
Asset Property: key_analyses>>pk_suggested
Asset Property: metadata_enrichment_info>>MDE_instrumented
Asset Property: term_assignment_profile>>attachment_id
Asset Property: term_assignment_profile>>completed_date
Asset Property: term_assignment_profile>>messages
Asset Property: term_assignment_profile>>messages
Asset Property: term_assignment_profile>>messages
Asset Property: term_assignment_profile>>semexp_completed_date
Asset Property: term_assignment_profile>>semexp_messages
Asset Property: term_assignment_profile>>semexp_start_date
Asset Property: term_assignment_profile>>semexp_status
Asset Property: term_assignment_profile>>start_date
Asset Property: term_assignment_profile>>status
Can't create a connection if you're including a reference connection
Applies to: 5.1.0 and later
When you're adding connections that contain references to catalogs, you might see the following error:
Unable to create connection</br>An unexpected error occurred of type Null pointer error. No further error information is available.
Workaround: Reference connections are not supported. Ensure that the platform connection doesn't contain any reference connections.
Migrating data source definitions from the Platform assets catalog will fail
Applies to: 5.1.0 and later
Data source definitions can not be migrated and attempts to migrate data source definitions will cause the migration to fail.
Workaround: There is currently no workaround for this issue.
You can migrate all other content from the Platform assets catalog without issues.
Can't profile tables with clob and blob data types
Applies to: 5.1.1
You can't profile data for tables with the clob
or blob
data types in PostgreSQL catalog in watsonx.data. Instead, you can see a Statement cannot be processed message
.
Governance artifacts issues
You might encounter these known issues and restrictions when you use governance artifacts.
Error Couldn't fetch reference data values
shows up on screen after publishing reference data
Applies to: 5.1
When new values are added to a reference data set, and the reference data set is published, the following error is displayed when you try to click on the values:
Couldn't fetch reference data values. WKCBG3064E: The reference_data_value for the reference_data which has parentVersionId: <ID> and code: <code> does not exist in the glossary. WKCBG0001I: Need more help?
When the reference data set is published, the currently displayed view changes to Draft-history as marked by the green label on the top. The Draft-history view does not allow to view the reference data values.
Workaround: To view the values, click Reload artifact so that you can view the published version.
Publishing large reference data sets fails with Db2 transaction log full
Applies to: 5.1.0 and later
Publishing large reference data sets might fail with a Db2 error such as:
The transaction log for the database is full. SQLSTATE=57011
Workaround: Publish the set in smaller chunks, or increase Db2 transaction log size as described in the following steps.
-
Modify the transaction log settings with the following commands:
db2 update db cfg for bgdb using LOGPRIMARY 5 --> default value, should not be changed db2 update db cfg for bgdb using LOGSECOND 251 db2 update db cfg for bgdb using LOGFILSIZ 20480
-
Restart Db2.
You can calculate the required transaction log size as follows:
(LOGPRIMARY + LOGSECOND) * LOGFILSIZ
For publishing large sets, the following Db2 transaction log sizes are recommended:
- 5GB for 1M reference data values and 300K relationships
- 20GB for 1M reference data values and 1M relationships
- 80GB for 1M reference data values and 4M relationships
where the relationship count is the sum of the parent, term and value mapping relationships for reference data values in the set.
Imported data assets with assigned out-of-the-box data classes or terms have incorrect identifiers resulting in no enforcement of data protection rules
When you migrate data assets across Cloud Pak for Data instances and these assets have out-of-the-box data classes or terms assigned, the imported data assets indicate correct data class or term assignments but the assigned artifact ID is incorrect. As a result, any operations that reference the data class or term, such as data protection rules, can't be applied to the imported data assets.
Relationships between catalog assets and out-of-the-box governance artifacts cannot be migrated correctly.
Applies to: All versions of Cloud Pak for Data beginning with 4.0 and later.
Workaround: none
Business terms remain after the semantic automation layer integration is deleted from IBM watsonx.data
Applies to: 5.1.0 and later
Business terms that were imported to IBM Knowledge Catalog for a semantic automation layer (SAL) integration in watsonx.data are not removed when the integration is deleted. This can result in duplicate business terms if a new SAL integration is subsequently enabled and the same or similar business terms are uploaded again.
Workaround: To avoid duplicate business terms, the cluster administrator or the user who originally created the SAL registration must manually delete all business terms that were imported for the SAL integration.
Importing artifacts from a zip file fails but reports success when category is created manually
If you create a category manually with the same name as the one in the zip import file and then try to import the file, the import might not be working as expected due to category ID conflict, even if success is reported.
Applies to: 5.1.0 and 5.1.1
Fixed in: 5.1.2
Workaround: Delete the manually created category and import the zip file again. For more information on how to avoid conflicts during import, see Importing all governance artifacts from an instance with a ZIP file.
A custom property is still visible after restricting it to categories
Applies to: 5.1.2
When adding a restriction by a category to an already existing custom property with values added, it will be still visible for artifacts in categories it wasn’t restricted to. If there was not value added, the custom property won’t be visible.
Metadata import issues
You might encounter these known issues when you work with metadata import.
Assets are not imported from the IBM Cognos Analytics source when the content language is set to Japanese
Applies to: 5.1.0 and later
If you want to import metadata from the Cognos Analytics connection, where the user's content language is set to Japanese, no assets are imported. The issue occurs when you create a metadata import with the Get BI report lineage goal.
Workaround: In Cognos Analytics, change the user's content language from Japanese to English. Find the user for which you want to change the language, and change this setting in the Personal tab. Run the metadata import again.
When you import a project from a .zip file, the metadata import asset is not imported
Applies to: 5.1.0 and later
When you import a project from a file, metadata import assets might not be imported. The issue occurs when a metadata import asset was imported to a catalog, not to a project, in the source system from which the project was exported. This catalog does not exist on the target system and the metadata import asset can't be accessed.
Workaround: After you import the project from a file, duplicate metadata import assets and add them to a catalog that exists on the target system. For details, see Duplicating a metadata import asset.
Lineage metadata cannot be imported from the Informatica PowerCenter connection
Applies to: 5.1.0 and later
When you import lineage metadata from the Informatica PowerCenter connection, the metadata job run fails with the following message:
400 [Failed to create discovery asset. path=/GLOBAL_DESEN/DM_PES_PESSOA/WKF_BCB_PES_PESSOA_JURIDICA_DIARIA_2020/s_M_PEJ_TOTAL_03_CARREGA_ST3_2020/SQ_FF_ACFJ671_CNAE_SECUND�RIA details=ASTSV3030E: The field 'name' should contain valid unicode characters.]",
"more_info" : null
Workaround: Ensure that the encoding value is the same in the workflow file in Informatica PowerCenter and in the connection that was created in Automatic Data Lineage. If the values are different, use the one from the Informatica
PowerCenter workflow file.
To solve the issue, complete these steps:
-
Open Automatic Data Lineage:
https://<CPD-HOSTNAME>/manta-admin-gui/
-
Go to Connections > Data Integration Tools > IFPC and select the connection for which the metadata import failed.
-
In the Inputs section, change the value of the Workflow encoding parameter to match the value from the Informatica PowerCenter workflow file.
-
Save the connection.
-
In IBM Knowledge Catalog, reimport assets for the metadata import that failed.
Related assets are not displayed in lineage that was created from the Greenplum connection
Applies to: 5.1.1
Fixed in: 5.1.2
When you import lineage metadata from the Greenplum connection by using the Get lineage or Get ETL lineage option, related assets are not included in the lineage.
Dummy assets get created for any file assets that come from Amazon S3 to show the complete business data lineage if Get ETL job lineage is performed
Applies to: 5.1.0 and later
If you perform Get ETL job lineage import involving Amazon S3 connection, dummy assets get created for any file assets that come from Amazon S3 connection to show the complete business data lineage. If you perform metadata import for the same Amazon S3 connection, a duplicate asset will get created for the dummy asset created from Get ETL job lineage import and a valid asset discovered during the metadata import.
Cannot delete added inputs from a file
Applies to: 5.1.1
Fixed in: 5.1.2
After creating metadata import, the input from a file in the Select scope and source section cannot be deleted.
Metadata import jobs might fail for any Generic JDBC connection that uses a vaulted JDBC URL
Applies to: 5.1.0 and later
When you're importing metadata from any Generic JDBC connection that uses a vaulted JDBC URL, which you enabled by using Enabling users to use JDBC URLs stored in secrets, your metadata import job fails.
Workaround: After the reconciliation process finishes, restart wdp-connect-flight
pod manually.
SocketTimeoutException during metadata import
Applies to: 5.1.2
During metadata import, when records from a CSV file that contains more than 30,000 rows are read, SocketTimeoutException
is returned. This indicates a network issue where the connection between the client and server was unexpectedly
closed.
Workaround:
-
Log in to the OpenShift console.
-
Go to Workloads > Pods > metadata-discovery-pod.
-
Go to the Environment section.
-
Search for the
manta_wf_export_download
environment variable and set it to true.Example:
manta_wf_export_download=true
By setting the variable, you're bypassing the socket timeout issue and downloading the CSV file to the local system. As a result, the CSV file can be read locally rather than over the network. After the CSV file is read, the locally downloaded file is deleted from the local system.
Metadata enrichment issues
You might encounter these known issues when you work with metadata enrichment.
Running primary key or relations analysis doesn't update the enrichment and review statuses
Applies to: 5.1.0 and later
The enrichment status is set or updated when you run a metadata enrichment with the configured enrichment options (Profile data, Analyze quality, Assign terms). However, the enrichment status is not updated when you run a primary key analysis or a relationship analysis. In addition, the review status does not change from Reviewed to Reanalyzed after review if new keys or relationships were identified.
Writing metadata enrichment output to an earlier version of Apache Hive than 3.0.0
Applies to: 5.1
If you want to write data quality output generated by metadata enrichment to an Apache Hive database at an earlier software version than 3.0.0, set the following configuration parameters in your Apache Hive Server:
set hive.support.concurrency=true;
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
set hive.enforce.bucketing=true; # not required for version 2
set hive.compactor.initiator.on=true;
set hive.compactor.cleaner.on=true; # might not be available depending on the version
set hive.compactor.worker.threads=1;
For more information, see Hive Transactions.
Issues with the Microsoft Excel add-in
Applies to: 5.1.0 and later
The following issues are known for the Review metadata add-in for Microsoft Excel:
-
When you open the drop-down list to assign a business term or a data class, the entry Distinctive name is displayed as the first entry. If you select this entry, it shows up in the column but does not have any effect.
-
Updating or overwriting existing data in a spreadsheet is currently not supported. You must use an empty template file whenever you retrieve data.
-
If another user works on the metadata enrichment results while you are editing the spreadsheet, the other user's changes can get lost when you upload the changes that you made in the spreadsheet.
-
Only assigned data classes and business terms are copied from the spreadsheet columns Assigned / suggested data classes and Assigned / suggested business terms to the corresponding entry columns. If multiple business terms are assigned, each one is copied to a separate column.
Republishing doesn't update primary key information in catalog
Applies to: 5.1.0 and later
If you remove primary key information from a data asset that initially was published with the primary key information to a catalog with the duplicate-asset handling method Overwrite original assets in the metadata enrichment results and then republish the asset to that catalog, the primary key information on the catalog asset remains intact.
Workaround: Delete the existing catalog asset before you republish the data asset from the metadata enrichment results.
Publishing SQL-based data assets from the metadata enrichment results isn't blocked
Applies to: 5.1.0 and later
Although publishing SQL-based dynamic view to catalogs is not supported, the action is not blocked in the metadata enrichment results. If you publish a dynamic view from the metadata enrichment results, the asset becomes available in the catalog with read-only SQL information.
Updates of business term abbreviations are not considered in name generation
Applies to: 5.1.0
Fixed in: 5.1.1
If abbreviations of business terms are updated after the initial load of the library used for fuzzy matching in metadata enrichment, the updates are not considered for name expansion in later runs of metadata enrichment.
Workaround: None.
Masked data might be profiled when the data source is IBM watsonx.data
Applies to: 5.1.0 and later
If a user who is not the owner of a protected data asset in IBM watsonx.data adds such asset to a project and runs metadata enrichment on it, the masked data is sent for profiling. As a result, even the asset owner will see the profile with masked data.
Workaround: None.
Data quality issues
You might encounter these known issues when you work with data quality assets.
Rules with multiple joins might return incorrect results for data assets from Apache Cassandra, Apache Hive, MongoDB, or Oracle data sources
Applies to: 5.1.0
Fixed in: 5.1.1
A data quality rule that is created from one or more data quality definitions and contains multiple joins might return incorrect results when is is run on data assets from Apache Cassandra, Apache Hive, MongoDB, or Oracle data sources that are connected through a Generic JDBC connection.
Workaround: Use the respective native connector.
Rules bound to columns of the data type NUMERIC in data assets from Oracle data sources might not work
Applies to: 5.1.0 and later
Testing or running a data quality rule that is bound to a NUMERIC column in a data asset from an Oracle data source fails if the data source is connected through a Generic JDBC connection.
Workaround: Use the native connector.
Runs of migrated data quality rules complete with warnings
Applies to: 5.1.0 and later
When you run a data quality rule that was migrated from the legacy data quality feature or from InfoSphere Information Server, you might see the message Run successful with warnings
.
Workaround: None. You can ignore such warnings.
Data quality output settings show some unsupported parameters
Applies to: 5.1.2
When you configure an output table for data quality rules, the user interface accepts some parameters for the table name that are currently not supported:
#execution_id#
#job_id#
#job_run_id#
When you use these parameters in your table configuration, they are not dynamically replaced with the corresponding values but treated as literal values when the table is created.
Workaround: None. Do not use these parameters.
MANTA Automated Data Lineage
You might encounter these known issues and restrictions when MANTA Automated Data Lineage is used for capturing lineage.
Metadata import jobs for getting lineage might take very long to complete
Applies to: 5.1.0 and later
If multiple lineage scans are requested at the same time, the corresponding metadata import jobs for getting lineage might take very long to complete. This is due to the fact that MANTA Automated Data Lineage workflows can't run in parallel but are executed sequentially.
Chrome security warning for Cloud Pak for Data deployments where MANTA Automated Data Lineage for IBM Cloud Pak for Data is enabled
Applies to: 4.8.0 and later
When you try to access a Cloud Pak for Data cluster that has MANTA Automated Data Lineage for IBM Cloud Pak for Data enabled from the Chrome web browser, the message Your connection is not private
is displayed and you can't proceed.
This is due to MANTA Automated Data Lineage for IBM Cloud Pak for Data requiring an SSL certificate to be applied and occurs only if a self-signed certificate is used.
Workaround: To bypass the warning for the remainder of the browser session, type thisisunsafe
anywhere on the window. Note that this code changes every now and then. The mentioned code is valid as of the date
of general availability of Cloud Pak for Data 4.6.0. You can search the web for the updated code if necessary.
Columns are displayed as numbers for a DataStage job lineage in the catalog
Applies to: 5.1
The columns for a lineage that was imported from a DataStage job are not displayed correctly in the catalog. Instead of column names, column numbers are displayed. The issue occurs when the source or target of a lineage is a CSV file.
MANTA Automated Data Lineage will not function properly on IBM Knowledge Catalog Standard
Applies to: 5.1
If you install MANTA Automated Data Lineage when you have IBM Knowledge Catalog Standard installed as the prerequisite, MANTA will not function properly.
If you want to install MANTA, you will need to have IBM Knowledge Catalog Premium installed.
Not all stages are displayed in technical data lineage graph for the imported DataStage ETL flow
Applies to: 5.1
When you import a DataStage ETL flow and view it in the technical data lineage graph, only three stages are displayed, even when four stages were imported.
Workaround: By default, three connected elements are displayed in the graph. To display more elements, click the expand icon on the last or the first displayed element on the graph.
The Unlock connection with personal credentials
option isn't available when you're importing metadata for data assets
The option to unlock a connection with personal credentials when you're importing metadata for data assets isn't available. Instead, you can see the No assets found
message.
Applies to: 5.1.1
Fixed in: 5.1.2
Workaround:
- Go to the Asset tab on the project page.
- In Asset types, identify the connection that you want to specify in your metadata import, and click it.
- In the Credentials section, type your username and password and save the changes.
- Create a metadata import.
Now, the option to unlock connection with personal credentials is available.
Can't Get lineage
with a DB2 connection in FIPS environments
Applies to: 5.1.0 and later
If you try to import metadata for data assets with the Get lineage
scenario with a DB2 connection in a FIPS environment, the metadata import fails and the following error message is displayed.
Error creating the metadata import Metadata import creation failed due to connection validation errors. Not all connections in the metadata import passed validation. Check the log for the complete validation results.
Business lineage issues
You might encounter these known issues and restrictions with lineage.
Lineage metadata don’t show on Knowledge Graph after upgrading
Applies to: 5.1.0 and later
After upgrading to 5.1.0 and later, an unknown error appears on the lineage tab.
Workaround: To start seeing the Knowledge Graph, you need to resync catalogs' metadata, see Resync of lineage metadata.
Business data lineage is incomplete for the metadata imports with Get ETL job lineage or Get BI report lineage goals
Applies to: 5.1
In some cases, when you display business lineage between databases and ETL jobs or BI reports, some assets are missing, for example, a starting database. The data was imported by using the Get ETL job lineage or Get BI report lineage import option. Technical data lineage correctly shows all assets.
Workaround: Sometimes MANTA Automated Data Lineage cannot map the connection information from an ETL job or a BI report to the existing connections in IBM Knowledge Catalog. Follow these steps to solve the issue:
-
Open the MANTA Automated Data Lineage Admin UI:
https://<CPD-HOSTNAME>/manta-admin-gui/
-
Go to Log Viewer and from the Source filter select Workflow Execution.
-
From the Workflow Execution filter, select the name of the lineage workflow that is associated with the incomplete business lineage.
-
Look for the
dictionary_manta_mapping_errors
issue category and expand it. -
In each entry, expand the error and click View Log Details.
-
In each error details, look for the value of
connectionString
. For example, in the following error message, the value of theconnectionString
parameter isDQ DB2 PX
.2023/11/14 18:40:12.186 PM [CLI] WARN - <provider-name> [Context: [DS Job 2_PARAMETER_SET] flow in project [ede1ab09-4cc9-4a3f-87fa-8ba1ea2dc0d8_lineage]] DICTIONARY_MANTA_MAPPING_ERRORS - NO_MAPPING_FOR_CONNECTION User message: Connection in use could not be automatically mapped to one of the database connections configured in MANTA. Technical message: There is no mapping for the connection Connection [type=DB2, connectionString=DQ DB2 PX, serverName=dataquack.ddns.net, databaseName=cpd, schemaName=null, userName=db2inst1]. Solution: Identify the particular database technology DB2 leading to "DQ DB2 PX" and configure it as a new connection or configure the manual mapping for that database technology in MANTA Admin UI. Lineage impact: SINGLE_INPUT
-
Depending on the connection that you used for the metadata import, go to Configuration > CLI > connection server > connection server Alias Mapping, for example DB2 > DB2 Alias Mapping.
-
Select the connection used in workflow and click Full override.
-
In the Connection ID field, add the value of the
connectionString
parameter that you found in the error details, for exampleDQ DB2 PX
. -
Rerun the metadata import job in IBM Knowledge Catalog.
Hops for components and columns of components work only inside the data integration flow area of an expanded job node
Applies to: 5.0.1 and later
When working with the lineage graph, hops for components and columns of data integration components work only inside the data integration flow area of an expanded job node and don't connect columns of nodes outside of the flow area.
Cannot import lineage mappings
Applies to: 5.1.1
Fixed in: 5.1.2
When importing lineage mappings to a catalog, the import results with an error Import asset <import-asset-id> does not have a status object.
Relationship explorer issues
You might encounter these known issues and restrictions with relationship explorer.
Relationship explorer is not working after installing the IBM Software Hub Version 5.1.0
Applies to: 5.1 and later
The relationship explorer feature cannot be accessed after installing IBM Knowledge Catalog and IBM Manta Data Lineage on IBM Software Hub Version 5.1. when the value useFDB: False
was set in the install-options.yml
file.
Workaround: To resolve the issue, contact IBM Support.
Error Your flow is empty shows up on screen after clicking Explore relationships in the catalog
Applies to: 5.1.1
Fixed in: 5.1.2
All metadata import assets must have a data source definition assigned to a connection if the value useFDB: False
was set during enabling optional features. If data source definition was assigned to a connection after publishing
an asset to catalog, error Your flow is empty will show up on the screen when clicking Explore relationships.
Workaround: After assigning data source definition to a connection, republish the metadata import asset to the catalog. For more information about creating data source definition, see Creating a data source definition.
Cannot view project metadata import assets in the relationship explorer
Applies to: 5.1.1
Fixed in: 5.1.2
When selecting Explore relationships in a project for metadata import assets with a data source definition assigned to a connection, the Your flow is empty error shows up on the canvas.
Reporting issues
You might encounter these known issues and restrictions with BI reporting.
Reporting setup page shows an error when two users start reporting
When two reporting administrators try to start reporting at the same time, it is successful for the first user only, and an error Settings couldn't be saved
is displayed for the other user. When the reporting is already started,
the users should only be able to update the reporting settings.
Applies to: 5.1.1
Workaround: When the error is displayed, refresh the page to see the Update reporting button.
Limitations
Catalogs and projects
Duplicate actions fail if dynamic IP address are used
Applies to: 5.1
Duplicate actions work only for connections with static IP addresses. If the connection is using a hostname with a dynamic IP address, duplicate actions might fail during connection creation.
Long names of the asset owners get truncated when hovering over their avatars
Applies to: 5.1
When you are hovering over the avatar to show the long name of the asset owner in the side panel, the name gets truncated if it is longer than 40 characters or contains a space or a special character. If the name is longer than 40 characters, it will display correctly as long as it contains a space or '-' within the first 40 characters.
Can't add individual group members as asset members
Applies to: 5.1
You can't add individual group members as asset members. You can add individual group members as catalog collaborators and then as asset members.
Catalog asset search doesn't support special characters
Applies to: 5.1
If search keywords contain any of the following special characters, the search filter doesn't return the most accurate results.
Search keywords:
. + - && || ! ( ) { } [ ] ^ " ~ * ? : \
Workaround: To obtain the most accurate results, search only for the keyword after the special character. For example, instead of AUTO_DV1.SF_CUSTOMER
, search for SF_CUSTOMER
.
Missing default catalog and predefined data classes
Applies to: 5.1
The automatic creation of the default catalog after installation of the IBM Knowledge Catalog service can fail. If it does, the predefined data classes are not automatically loaded and published as governance artifacts.
Workaround: Ask someone with the Administrator role to follow the instructions for creating the default catalog manually.
Special or double-byte characters in the data asset name are truncated on download
Applies to: 5.1
When you download a data asset with a name that contains special or double-byte characters from a catalog, these characters might be truncated from the name. For example, a data asset named special chars!&@$()テニス.csv
will
be downloaded as specialchars!().csv
.
The following character sets are supported:
- Alphanumeric characters:
0-9
,a-z
,A-Z
- Special characters:
! - _ . * ' ( )
Catalog UI does not update when changes are made to the asset metadata
Applies to: 5.1
If the Catalog UI is open in a browser while an update is made to the asset metadata, the Catalog UI page will not automatically update to reflect this change. Outdated information will continue to be displayed, causing external processes to produce incorrect information.
Workaround: After the asset metadata is updated, refresh the Catalog UI page at the browser level.
A blank page might be rendered when you search for terms while manually assigning terms to a catalog asset
Applies to: 5.1
When you search for a term to assign to a catalog asset and change that term while the search is running, it can happen that a blank page is shown instead of any search results.
Workaround: Rerun the search.
Project assets that are added while you create segmented data assets might not be available for selection
Applies to: 5.1
If assets are added to the project while you are viewing the list of data assets to pick the column for segmentation, these new assets are listed, but you cannot select them.
Workaround: Cancel the creation process and start anew.
An extra path to manage catalogs in the navigation menu
Applies to: 5.1
If you have the Manage catalogs user permission, an extra Administration > Catalogs path to manage catalogs shows up in the navigation menu.
Governance artifacts
Cannot use CSV to move data class between Cloud Pak for Data instances
Applies to: 5.0.0 and later
If you try to export data classes with matching method Match to reference data to CSV, and then import it into another Cloud Pak for Data instance, the import fails.
Workaround: For moving governance artifact data from one instance to another, especially data classes of this matching method, use the ZIP format export and import. For more information about the import methods, see Import methods for governance artifacts in the Cloud Pak for Data documentation.
Unable to use masked data in visualizations from data assets imported from version 4.8 or earlier
Applies to: 5.1.0
If you import data assets with masked data from version 4.8 or earlier into your project, you cannot use these assets to create visualizations. If you attempt to generate a chart in the Visualization tab of a data asset from
an imported asset that has masked data, the following error message is received: Bad Request: Failed to retrieve data from server. Masked data is not supported
.
Workaround: To properly mask data with imported data assets in visualization, you must configure your platform with Data Virtualization as a protection solution. For more information, see the Data Virtualization as a protection solution section of the Protection solutions for data source definitions topic.
Metadata import
Metadata import jobs might be stuck due to issues related to RabbitMQ
Applies to: 5.1
If the metadata-discovery
pod starts before the rabbitmq pods are up after a cluster reboot, metadata import jobs can get stuck while attempting to get the job run logs.
Workaround: To fix the issue, complete the following steps:
- Log in to the OpenShift console by using admin credentials.
- Go to Workloads > Pods.
- Search for rabbitmq.
- Delete the
rabbitmq-0
,rabbitmq-1
, andrabbitmq-2
pods. Wait for the pods to be back up and running. - Search for discovery.
- Delete the
metadata-discovery
pod. Wait for the pod to be back up and running. - Rerun the metadata import job.
Data assets might not be imported when running an ETL job lineage import for DataStage flows
Applies to: 5.1
When you create and run a metadata import with the goal Get ETL job lineage where the scope is determined by the Select all DataStage flows and their dependencies in the project option, data assets from the connections associated with the DataStage flows are not imported.
Workaround: Explicitly select all DataStage flows and connections when you set the scope instead of using the Select all DataStage flows and their dependencies in the project option.
When lineage import fails, no details in the job log are available
Applies to: 5.1.0
When you import lineage metadata and the import of lineage data fails, the job log files do not contain any details and explanation about why the import failed.
Workaround: Look for details in the scanner
pod or scanner-worker
pod log files.
When a job for importing lineage metadata hangs, it cannot be stopped
Applies to: 5.1.0
When you run a lineage metadata import and the job stops responding, the job can't be stopped.
Only files with .sql extension can be provided as manual input for metadata import from the Oracle and PostgreSQL sources
Applies to: 5.1.0
When you import metadata from the Oracle and PostgreSQL sources, only .sql
files can be used as manual input. Other formats like files with .pck
extension can't be used. This limitation is applicable when you install
IBM Manta Data Lineage.
Metadata enrichment
In some cases, you might not see the full log of a metadata enrichment job run in the UI
Applies to: 5.1
If the list of errors in a metadata enrichment run is exceptionally long, only part of the job log might be displayed in the UI.
Workaround: Download the entire log and analyze it in an external editor.
Schema information might be missing when you filter enrichment results
Applies to: 5.1
When you filter assets or columns in the enrichment results on source information, schema information might not be available.
Workaround: Rerun the enrichment job and apply the Source filter again.
Profiling in catalogs, projects, and metadata enrichment might fail for Teradata connections
Applies to: 5.1
If a Generic JDBC connection for Teradata exists with a driver version before 17.20.00.15, profiling in catalogs and projects, and metadata enrichment of data assets from a Teradata connection fails with an error message similar to the following one:
2023-02-15T22:51:02.744Z - cfc74cfa-db47-48e1-89f5-e64865a88304 [P] ("CUSTOMERS") - com.ibm.connect.api.SCAPIException: CDICO0100E: Connection failed: SQL error: [Teradata JDBC Driver] [TeraJDBC 16.20.00.06] [Error 1536] [SQLState HY000] Invalid connection parameter name SSLMODE (error code: DATA_IO_ERROR)
Workaround: For this workaround, users must be enabled to upload or remove JDBC drivers. For more information, see Enable users to upload, delete, or view JDBC drivers.
Complete these steps:
- Go to Data > Connectivity > JDBC drivers and delete the existing JAR file for Teradata (
terajdbc4.jar
). - Edit the Generic JDBC connection, remove the selected JAR files, and add
SSLMODE=ALLOW
to the JDBC URL.
For assets from SAP OData sources, the metadata enrichment results do not show the table type
Applies to: 5.1
In general, metadata enrichment results show for each enriched data asset whether the asset is a table or a view. This information cannot be retrieved for data assets from SAP OData data sources and is thus not shown in the enrichment results.
Data quality
Rules run on columns of type timestamp with timezone fail
Applies to: 5.1
The data type timestamp with timezone is not supported. You can't apply data quality rules to columns with that data type.
Business lineage
An unnecessary edge appears when expanding data integration assets
Applies to: 5.1.0 and later
After expanding a data integration asset and clicking Show next or Show all, the transformer nodes will have an unnecessary edge that points to themselves.
Parent topic: Service issues