What's new and changed in Watson Knowledge Catalog

Installing or upgrading Watson Knowledge Catalog

Refresh 9 of Cloud Pak for Data Version 4.0

A new version of Watson Knowledge Catalog was released in May 2022.

Operand version: 4.0.9

This release includes the following changes:

Bug fixes

Catalogs

Issue: Users with the viewer role cannot edit personal connection credentials in a catalog.

Issue: Catalog admins or editors, who are asset members or owners, are not able to edit data asset metadata in a catalog when it is locked by a personal connection.

Issue: Data profiling fails for connected assets that are virtualized and for connected assets that use the S3 or COS connector types.

Governance artifacts

Issue: Can't publish reference data after renaming or editing the effective date.

Security fixes

This release includes fixes for the following security issues:

CVE-2018-1313

CVE-2019-17566, CVE-2019-20633

CVE-2020-11988, CVE-2020-13956, CVE-2020-15250, CVE-2020-26555, CVE-2020-28500, CVE-2020-8203,

CVE-2021-20320, CVE-2021-20322, CVE-2021-21781, CVE-2021-23337, CVE-2021-23841, CVE-2021-28168, CVE-2021-33624, CVE-2021-34556, CVE-2021-34693, CVE-2021-34866, CVE-2021-3490, CVE-2021-34981, CVE-2021-35477, CVE-2021-3612, CVE-2021-36221, CVE-2021-3640, CVE-2021-3655, CVE-2021-3669, CVE-2021-3743, CVE-2021-3744, CVE-2021-3752, CVE-2021-3753, CVE-2021-3759, CVE-2021-3764, CVE-2021-3772, CVE-2021-3773, CVE-2021-38166, CVE-2021-38198, CVE-2021-38199, CVE-2021-38206, CVE-2021-38297, CVE-2021-3864, CVE-2021-39293, CVE-2021-4028, CVE-2021-4037, CVE-2021-40490, CVE-2021-4148, CVE-2021-4157, CVE-2021-41771, CVE-2021-41772, CVE-2021-41864, CVE-2021-4197, CVE-2021-4203, CVE-2021-42739, CVE-2021-43056, CVE-2021-43389, CVE-2021-43976, CVE-2021-44716, CVE-2021-44906, CVE-2021-45261, CVE-2021-45346, CVE-2021-45485

CVE-2022-0286, CVE-2022-0322, CVE-2022-0778, CVE-2022-1199, CVE-2022-1204, CVE-2022-1205, CVE-2022-21704, CVE-2022-24329, CVE-2022-24771, CVE-2022-24772, CVE-2022-24773, CVE-2022-25636, CVE-2022-27191, CVE-2022-28796

Refresh 8 of Cloud Pak for Data Version 4.0

A new version of Watson Knowledge Catalog was released in April 2022.

Operand version: 4.0.8

This release includes the following changes:

New features

The 4.0.8 release of Watson Knowledge Catalog includes the following features and updates:

Filtering workflow tasks

In your Task inbox, you can now filter your open, in progress, and completed tasks so that you can find the tasks you want faster. Several filters are available, including request type and task status.

Bug fixes

Catalogs

Issue: When migrating catalog assets from one IBM Cloud Pak for Data installation to another, error message java.lang.NoClassDefFoundError: okhttp3.RequestBody is issued.

Data curation

Issue: Operator names are treated as case-sensitive by the data quality rule definitions rule builder.

General

Issue: For unstructured data sources, content from subfolders isn't imported.

Issue: Timeout error might occur for successful imports if multiple metadata import jobs are handling many simultaneous and consecutive assets.

Issue: When importing data assets to a project, the assets are added as CSV files.

Issue: Scaling up Watson Knowledge Catalog results in higher CPU consumption for the reporting service.

Governance artifacts

Issue: Rejected term assignment is visible from the business term details view in IGC.

Reporting

Issue: Operator names are treated as case-sensitive by the data quality rule definitions rule builder.

Security fixes

This release includes fixes for the following security issues:

CVE-2015-8315

CVE-2018-11793, CVE-2018-3745, CVE-2018-8023

CVE-2019-0205, CVE-2019-16866, CVE-2019-25033, CVE-2020-11979

CVE-2020-13949, CVE-2020-1945, CVE-2020-26939, CVE-2020-7598

CVE-2021-0920, CVE-2021-22060, CVE-2021-23463, CVE-2021-28170, CVE-2021-3521, CVE-2021-35550, CVE-2021-35556, CVE-2021-35559, CVE-2021-35561, CVE-2021-35564, CVE-2021-35565, CVE-2021-35567, CVE-2021-35578, CVE-2021-35586, CVE-2021-35588, CVE-2021-35603, CVE-2021-3711, CVE-2021-3807, CVE-2021-3872, CVE-2021-39031, CVE-2021-3984, CVE-2021-4019, CVE-2021-4122, CVE-2021-4154, CVE-2021-4192, CVE-2021-4193, CVE-2021-42392, CVE-2021-44521, CVE-2021-44964

CVE-2022-0261, CVE-2022-0318, CVE-2022-0330, CVE-2022-0359, CVE-2022-0361, CVE-2022-0392, CVE-2022-0413, CVE-2022-0435, CVE-2022-0487, CVE-2022-0492, CVE-2022-0516, CVE-2022-0635, CVE-2022-0639, CVE-2022-0644, CVE-2022-0667, CVE-2022-0686, CVE-2022-0691, CVE-2022-0839, CVE-2022-0907, CVE-2022-21248, CVE-2022-21282, CVE-2022-21283, CVE-2022-21293, CVE-2022-21294, CVE-2022-21296, CVE-2022-21299, CVE-2022-21305, CVE-2022-21340, CVE-2022-21341, CVE-2022-21360, CVE-2022-21365, CVE-2022-21724, CVE-2022-22942, CVE-2022-22965, CVE-2022-23221, CVE-2022-24407, CVE-2022-25258

Refresh 7 of Cloud Pak for Data Version 4.0

A new version of Watson Knowledge Catalog was released in March 2022.

Operand version: 4.0.7

This release includes the following changes:

Bug fixes

Catalogs

Issue: Asset preview page shows column data when returning to the Asset tab when the column is masked.

Resolution: The issue is fixed now.

Issue: Users who have Editor or Admin access via group membership cannot publish to catalog, 

Resolution: The issue is fixed now.

Issue: Data preview of Parquet files compressed with gz format from a Cloud Object Storage connection does not work.

Resolution: The issue is fixed now.

Issue: Users in a group with Platform Connections Admin rights cannot add users, delete users, or modify user access.       

Resolution: The issue is fixed now.

Data curation

Issue: Quick scan ignores tables where the table nickname or alias is specified instead of the actual table name.

Resolution: The issue is fixed now.

Issue: Automatic term assignment in automated discovery does not ignore rejected terms for similar columns when discovering new tables.

Resolution: The issue is fixed now.

Issue: Data quality analysis fails on a virtualized Db2® Big SQL table.

Resolution: The issue is fixed now.

Issue: In discovery jobs, you cannot use SSL connections with certificates from a vault that are configured with the secret type Key.

Resolution: The issue is fixed now.

Issue: Quick scan results cannot be filtered by data classes or terms.

Resolution: The issue is fixed now.

Issue: Publishing a quick scan connection to a catalog fails when there are over 100 platform connections.

Resolution: The issue is fixed now.

Issue: Deleting or updating a platform connection that is being used in data discovery does not delete or update the connection in data discovery.

Resolution: The issue is fixed now.

Issue: Long business term and table names are improperly displayed in the Data assets in a data quality project.

Resolution: The issue is fixed now.

Issue: Assets discovered from a Microsoft Azure Data Lake Store source and synced to the default catalog can have an incorrect source data connection associated.

Resolution: The issue is fixed now.

Issue: Selected sample size not honored when running quick scan on a Google BigQuery data source.

Resolution: The issue is fixed now.

General

Issue: Viewer, Editor, and Admin user roles in Platform Connections cannot see groups in the access control list.

Resolution: The issue is fixed now.

Governance artifacts

Issue: Some of the reference data does not appear when you try to pick a matching method when creating a data class.

Resolution: The issue is fixed now.

Issue: Workflow bell notifications do not work.

Resolution: The issue is fixed now.

Reporting

Issue: Can't unlock personal connection when setting up reporting.

Resolution: The issue is fixed now.

Security fixes

This release includes fixes for the following security issues:

CVE-2012-5784

CVE-2014-3596

CVE-2015-0886

CVE-2018-8032

CVE-2019-0227

CVE-2020-7608, CVE-2020-7774

CVE-2021-23555, CVE-2021-23566, CVE-2021-32796, CVE-2021-3408, CVE-2021-38185, CVE-2021-3918, CVE-2021-3981, CVE-2021-42550, CVE-2021-43519, CVE-2021-43859

CVE-2022-0122, CVE-2022-0512, CVE-2022-0536, CVE-2022-0554, CVE-2022-0563, CVE-2022-21271, CVE-2022-21668, CVE-2022-23181

Refresh 6 of Cloud Pak for Data Version 4.0

A new version of Watson Knowledge Catalog was released in February 2022.

Operand version: 4.0.6

This release includes the following changes:

New features

The 4.0.6 release of Watson Knowledge Catalog includes the following features and updates:

Access more data with new connection types

In Watson Knowledge Catalog, you can now work with data from these data sources:

• Generic S3 (Use to connect to a storage service that is compatible with the Amazon S3 API)
• Exasol

Data Scientist role has Access governance artifacts permission

With the Access governance artifacts permission, data scientists can see the details of governance artifacts that are assigned to assets to better understand the data. For details, see Predefined roles and permissions.

Keeping catalog names unique

When you create a catalog in the Create a catalog page, you must now use a unique name. Unique catalog names will avoid ambiguity problems and sync errors. If you need to use a duplicate name for a catalog, use the API to rename or create a catalog.

Generating reports on Watson Knowledge Catalog data

Now you can get insights into your catalogs, projects, and governance artifacts by setting up reporting for Watson Knowledge Catalog. The data is sent to an external database where you can run SQL queries to generate reports. For details on configuring the reporting, see Reporting on Watson Knowledge Catalog data.

Enhanced linguistic matching in quick scan

In quick scan, term assignment based on linguistic name matching now provides better results because the longest common substrings of term and asset names are considered.

Bug fixes

Data curation

Issue: Db2 Warehouse connections with API key authentication don't work in automated discovery.

Resolution: API key authentication for Db2 Warehouse connections now works in automated discovery.

Issue: Quick scan assets table only shows 10 rows when Items per page is set to 50.

Resolution: Quick scan assets table now shows 50 items per page.

Issue: Business term list items overlap while assigning business terms to discovered columns.

Resolution: Business term list items are now displayed properly.

Issue: Business term search for quick scan discovery results does not work when the search term includes a space.

Resolution: Business term search will now work when the search term includes a space.

Issue: Connections created in a default catalog do not work for automated discovery.

Resolution: Connections created in a default catalog will now work for automated discovery.

Issue: Apache Hive connection fails to be added in discover page though Db2 vault connection succeeds.

Resolution: Apache Hive connection will now behave the same as a Db2 vault connection.

Data discovery

Issue: Bulk business term assignment takes a long time to complete.

Resolution: Bulk business term assignment time has been reduced.

Data quality

Issue: Schema names containing a hyphen in MongoDB cause the column analysis and data quality analysis to fail.

Resolution: Column analysis and data quality analysis no longer fail when schema name contains a hyphen.

Issue: Unable to view details for the "Duplicated values" dimension at column level.

Resolution: You can now view details for the "Duplicated values" dimension at column level.

Issue: Best matches for business terms are not prioritized in search results.

Resolution: Business term search results are now sorted by best match.

Issue: No warning is given when the project owner is being deleted from a project.

Resolution: You will now get a warning before attempting to delete the project owner.

Governance artifacts

Issue: Activity log doesn't load in imported categories.

Resolution: Activities for categories and reference data sets are now displayed as expected.

Import

Issue: When an .isx project from another environment is imported by using the istool command, the rules in that project can't be viewed or accessed from the Data rules tab.

Resolution: Rules in a data quality project that was imported from an .isx file can now be accessed.

Install

Issue: Deploy of solr job solr-configset-collection causes IIS reconcile to fail.

Resolution: solr-configset-collection is now skipped and will not cause a failure during reconcile.

Profiling

Issue: Profiling might fail for assets from connections that use credentials from a vault.

Resolution: You can now profile assets from connections that use credentials from a vault.

Roles

Issue: Available options on the Data assets tab do not immediately reflect a role change.

Resolution: Available options on the Data assets tab now reflect new roles immediately.

Security fixes

This release includes fixes for the following security issues:

CVE-2016-5397

CVE-2017-18640

CVE-2018-1320

CVE-2019-0205

CVE-2020-13936, CVE-2020-14060, CVE-2020-14061, CVE-2020-14062, CVE-2020-14195, CVE-2020-14422, CVE-2020-15257, CVE-2020-16135, CVE-2020-24616, CVE-2020-24750, CVE-2020-25649, CVE-2020-28362, CVE-2020-28366, CVE-2020-28367, CVE-2020-29652, CVE-2020-35490, CVE-2020-35491, CVE-2020-35728, CVE-2020-36179, CVE-2020-36180, CVE-2020-36181, CVE-2020-36182, CVE-2020-36183, CVE-2020-36184, CVE-2020-36185, CVE-2020-36186, CVE-2020-36187, CVE-2020-36188, CVE-2020-36189, CVE-2020-8492

CVE-2021-20190, CVE-2021-21334, CVE-2021-23490, CVE-2021-27918, CVE-2021-31525, CVE-2021-33194, CVE-2021-33195, CVE-2021-33196, CVE-2021-33197, CVE-2021-33198, CVE-2021-34141, CVE-2021-34558, CVE-2021-3842, CVE-2021-44832, CVE-2021-45931

CVE-2022-0155, CVE-2022-21676

Refresh 5 of Cloud Pak for Data Version 4.0

A new version of Watson Knowledge Catalog was released in January 2022.

Operand version: 4.0.5

This release includes the following changes:

Security fixes

This release includes fixes for the following security issues:

• CVE-2021-45105
• CVE-2021-45046

Refresh 4 of Cloud Pak for Data Version 4.0

A new version of Watson Knowledge Catalog was released in December 2021.

Operand version: 4.0.4

This release includes the following changes:

Bug fixes

Catalog

Issue: Profiling fails for synced file assets in the default catalog.

Resolution: Profiling now works for synced file assets in the default catalog.

Issue: If you use "replace" or "update" duplicate settings in your catalog and another asset in the catalog will be assigned these terms, on an attempt to add the same asset again, the asset will be deleted.

Resolution: Asset will not be deleted.

Connections

Issue: You can view or select from a maximum of 50 secrets when creating or editing a connection.

Resolution: You can now view or select more than 50 secrets when creating or editing a connection.

Data curation: quick scan

Issue: Quick scan job status shows Analyzing instead of Queued for analysis.

Resolution: Correct status is now displayed during a quick scan.

Issue: Sorting on date field does not respect calendar order in quick scan results.

Resolution: Sort order now respects calendar order.

Issue: Publishing more than 1,000 assets from the quick scan results fails and the assets remain in "Loading error" status.

Resolution: You can now publish more than 1,000 assets at once.

Data discovery

Issue: For SQL Server data sources, a SQL error is returned when running discovery on columns with names containing the '#' character.

Resolution: In SQL statements accessing SQL Server data sources, columns and tables are now enclosed in double quotes, so that names containing the '#' character don't cause errors.

Data discovery: usability

Issue: In the Google Chrome browser, when opening a large list of filter options in Quick scan results and selecting one of them, the content in the selection UI is moved off the display area.

Resolution: Large selection lists are now rendered properly in the Google Chrome browser.

Data quality

Issue: Testing a rule returns an incorrect result if the rule definition contains aggregations.

Resolution: An error message is now displayed if the definition of the rule being tested contains aggregations or reference functions.

Data refinery

Issue: Parquet files containing double-byte characters (such as Japanese and Chinese) cannot be previewed in projects.

Resolution: Parquet files containing double-byte characters can now be previewed in projects.

Governance artifacts

Issue: The Export button in a category is enabled for users with only view permission on the category.

Resolution: The Export button in a category is now disabled for users without permission to export categories and governance artifacts.

Issue: For reference data and governance rules, the Mark for deletion option is enabled for users without delete permission.

Resolution: The Mark for deletion option is now disabled for users without delete permission.

Issue: Users with only the view permission can see the history of changes of draft artifacts on the activities pane.

Resolution: Users with only the view permission can now see no activities but publish actions on the activities pane.

Issue: Not all custom attributes are displayed for categories after modifying one of the custom attributes.

Resolution: All custom attributes are now displayed after updating an attribute.

Issue: The Add subcategories permission is not sufficient to create subcategories.

Resolution: The Add subcategories permission now allows to add subcategories.

Issue: The Mark for deletion option is not available on the details page of the artifact for users with the required permission.

Resolution: Users with the appropriate permissions can now mark artifacts for deletion on the details page.

Profiling

Issue: The error message doesn't provide enough information if profiling fails because the job couldn't be created.

Resolution: The error message now provides information about the cause of the error.

Security fixes

This release includes fixes for the following security issues:

CVE-2011-5036

CVE-2012-6109

CVE-2013-0183, CVE-2013-0184, CVE-2013-0262, CVE-2013-0263

CVE-2014-6393

CVE-2015-3225

CVE-2016-10228, CVE-2016-2124, CVE-2016-4074

CVE-2017-13745, CVE-2017-15713, CVE-2017-5499, CVE-2017-5503, CVE-2017-5504, CVE-2017-5505, CVE-2017-9735, CVE-2017-9782

CVE-2018-0734, CVE-2018-1000021, CVE-2018-1109, CVE-2018-11766, CVE-2018-11767, CVE-2018-11771, CVE-2018-1296, CVE-2018-15594, CVE-2018-15919, CVE-2018-16471, CVE-2018-16487, CVE-2018-16862, CVE-2018-17977, CVE-2018-18873, CVE-2018-19057, CVE-2018-19139, CVE-2018-19416, CVE-2018-19517, CVE-2018-19539, CVE-2018-19540, CVE-2018-19541, CVE-2018-19542, CVE-2018-19543, CVE-2018-20570, CVE-2018-20622, CVE-2018-20845, CVE-2018-20847, CVE-2018-3721, CVE-2018-5407, CVE-2018-5727, CVE-2018-5785, CVE-2018-7273, CVE-2018-8029, CVE-2018-8043, CVE-2018-9055, CVE-2018-9252

CVE-2019-0201, CVE-2019-0210, CVE-2019-1010266, CVE-2019-10744, CVE-2019-12380, CVE-2019-12973, CVE-2019-13012, CVE-2019-13224, CVE-2019-13631, CVE-2019-13750, CVE-2019-13751, CVE-2019-14283, CVE-2019-14284, CVE-2019-14868, CVE-2019-15165, CVE-2019-15213, CVE-2019-15217, CVE-2019-15218, CVE-2019-15219, CVE-2019-15291, CVE-2019-1547, CVE-2019-15505, CVE-2019-1551, CVE-2019-1559, CVE-2019-1563, CVE-2019-15794, CVE-2019-15807, CVE-2019-16089, CVE-2019-16161, CVE-2019-16162, CVE-2019-16163, CVE-2019-16782, CVE-2019-17495, CVE-2019-17594, CVE-2019-17595, CVE-2019-18218, CVE-2019-18276, CVE-2019-18806, CVE-2019-18874, CVE-2019-19012, CVE-2019-19054, CVE-2019-19066, CVE-2019-19080, CVE-2019-19081, CVE-2019-19082, CVE-2019-19083, CVE-2019-19203, CVE-2019-19204, CVE-2019-19246, CVE-2019-19377, CVE-2019-19462, CVE-2019-19529, CVE-2019-19530, CVE-2019-19535, CVE-2019-19536, CVE-2019-19603, CVE-2019-19816, CVE-2019-19965, CVE-2019-20095, CVE-2019-2054, CVE-2019-20838, CVE-2019-25013, CVE-2019-2708, CVE-2019-3842, CVE-2019-3881, CVE-2019-5827, CVE-2019-6110, CVE-2019-9169

CVE-2020-0404, CVE-2020-10001, CVE-2020-10135, CVE-2020-10730, CVE-2020-10737, CVE-2020-10781, CVE-2020-11494, CVE-2020-11609, CVE-2020-12762, CVE-2020-13434, CVE-2020-13435, CVE-2020-13543, CVE-2020-13558, CVE-2020-13584, CVE-2020-13645, CVE-2020-13776, CVE-2020-13949, CVE-2020-13974, CVE-2020-14039, CVE-2020-14145, CVE-2020-14150, CVE-2020-14155, CVE-2020-14304, CVE-2020-14318, CVE-2020-14323, CVE-2020-14343, CVE-2020-14390, CVE-2020-14416, CVE-2020-1472, CVE-2020-15168, CVE-2020-15358, CVE-2020-15389, CVE-2020-16125, CVE-2020-18442, CVE-2020-1968, CVE-2020-1971, CVE-2020-24025, CVE-2020-24370, CVE-2020-24870, CVE-2020-24977, CVE-2020-25219, CVE-2020-25639, CVE-2020-25645, CVE-2020-25656, CVE-2020-25717, CVE-2020-26116, CVE-2020-26137, CVE-2020-26154, CVE-2020-26160, CVE-2020-26555, CVE-2020-27170, CVE-2020-27171, CVE-2020-27218, CVE-2020-27223, CVE-2020-27618, CVE-2020-27783, CVE-2020-27814, CVE-2020-27820, CVE-2020-27823, CVE-2020-27824, CVE-2020-27828, CVE-2020-27842, CVE-2020-27843, CVE-2020-27845, CVE-2020-27918, CVE-2020-28097, CVE-2020-28196, CVE-2020-28469, CVE-2020-28493, CVE-2020-28500, CVE-2020-28915, CVE-2020-29361, CVE-2020-29362, CVE-2020-29363, CVE-2020-29374, CVE-2020-29623, CVE-2020-35501, CVE-2020-36048, CVE-2020-36241, CVE-2020-36311, CVE-2020-36327, CVE-2020-3702, CVE-2020-8037, CVE-2020-8161, CVE-2020-8184, CVE-2020-8203, CVE-2020-8231, CVE-2020-8284, CVE-2020-8285, CVE-2020-8286, CVE-2020-8927, CVE-2020-9492, CVE-2020-9948, CVE-2020-9951, CVE-2020-9983

CVE-2021-0941, CVE-2021-1765, CVE-2021-1788, CVE-2021-1789, CVE-2021-1799, CVE-2021-1801, CVE-2021-1817, CVE-2021-1820, CVE-2021-1825, CVE-2021-1826, CVE-2021-1844, CVE-2021-1870, CVE-2021-1871, CVE-2021-20066, CVE-2021-20178, CVE-2021-20191, CVE-2021-20231, CVE-2021-20232, CVE-2021-20254, CVE-2021-20266, CVE-2021-20270, CVE-2021-20271, CVE-2021-20277, CVE-2021-20305, CVE-2021-20320, CVE-2021-20321, CVE-2021-20322, CVE-2021-21300, CVE-2021-21775, CVE-2021-21779, CVE-2021-21781, CVE-2021-21806, CVE-2021-22876, CVE-2021-22898, CVE-2021-22922, CVE-2021-22923, CVE-2021-22924, CVE-2021-22925, CVE-2021-22946, CVE-2021-22947, CVE-2021-23192, CVE-2021-23336, CVE-2021-23337, CVE-2021-23382, CVE-2021-23413, CVE-2021-23424, CVE-2021-23840, CVE-2021-25214, CVE-2021-26926, CVE-2021-26927, CVE-2021-27218, CVE-2021-27219, CVE-2021-27291, CVE-2021-27645, CVE-2021-28153, CVE-2021-28163, CVE-2021-28650, CVE-2021-28657, CVE-2021-28957, CVE-2021-29060, CVE-2021-29154, CVE-2021-29657, CVE-2021-29921, CVE-2021-29922, CVE-2021-30002, CVE-2021-30661, CVE-2021-30663, CVE-2021-30665, CVE-2021-30682, CVE-2021-30689, CVE-2021-30720, CVE-2021-30734, CVE-2021-30744, CVE-2021-30749, CVE-2021-30758, CVE-2021-30795, CVE-2021-30797, CVE-2021-30799, CVE-2021-31535, CVE-2021-3177, CVE-2021-31799, CVE-2021-31810, CVE-2021-3200, CVE-2021-32066, CVE-2021-3272, CVE-2021-3326, CVE-2021-33503, CVE-2021-33560, CVE-2021-33574, CVE-2021-33623, CVE-2021-33624, CVE-2021-33910, CVE-2021-33928, CVE-2021-33929, CVE-2021-33930, CVE-2021-33938, CVE-2021-3421, CVE-2021-3426, CVE-2021-3428, CVE-2021-3443, CVE-2021-3444, CVE-2021-3445, CVE-2021-3449, CVE-2021-3450, CVE-2021-34556, CVE-2021-3467, CVE-2021-34693, CVE-2021-3481, CVE-2021-34866, CVE-2021-3490, CVE-2021-34981, CVE-2021-3516, CVE-2021-3517, CVE-2021-3518, CVE-2021-3520, CVE-2021-3537, CVE-2021-3541, CVE-2021-35477, CVE-2021-35550, CVE-2021-35556, CVE-2021-35559, CVE-2021-35561, CVE-2021-35564, CVE-2021-35565, CVE-2021-35567, CVE-2021-35578, CVE-2021-35586, CVE-2021-35588, CVE-2021-35603, CVE-2021-3572, CVE-2021-3575, CVE-2021-3583, CVE-2021-35942, CVE-2021-36084, CVE-2021-36085, CVE-2021-36086, CVE-2021-36087, CVE-2021-3612, CVE-2021-36159, CVE-2021-3621, CVE-2021-36222, CVE-2021-3640, CVE-2021-3655, CVE-2021-3664, CVE-2021-3669, CVE-2021-3711, CVE-2021-37159, CVE-2021-3743, CVE-2021-3744, CVE-2021-3752, CVE-2021-3753, CVE-2021-3759, CVE-2021-3764, CVE-2021-3765, CVE-2021-3772, CVE-2021-3773, CVE-2021-37750, CVE-2021-3778, CVE-2021-3796, CVE-2021-3803, CVE-2021-38166, CVE-2021-38198, CVE-2021-38199, CVE-2021-38206, CVE-2021-39293, CVE-2021-4002, CVE-2021-4028, CVE-2021-40330, CVE-2021-40490, CVE-2021-41134, CVE-2021-41186, CVE-2021-41617, CVE-2021-41816, CVE-2021-41817, CVE-2021-41819, CVE-2021-41864, CVE-2021-42374, CVE-2021-42375, CVE-2021-42378, CVE-2021-42379, CVE-2021-42380, CVE-2021-42381, CVE-2021-42382, CVE-2021-42383, CVE-2021-42384, CVE-2021-42385, CVE-2021-42386, CVE-2021-42574, CVE-2021-42739, CVE-2021-42771, CVE-2021-43056, CVE-2021-43389, CVE-2021-43975, CVE-2021-43976, CVE-2021-44228

Refresh 3 of Cloud Pak for Data Version 4.0

A new version of Watson Knowledge Catalog was released in November 2021.

Operand version: 4.0.3

This release includes the following changes:

New features

The 4.0.3 release of Watson Knowledge Catalog includes the following features and updates:

Support for vaults

Watson Knowledge Catalog supports integration with an external vault where your secrets are stored. For more information, see Managing secrets and vaults.

You can use vaults and secrets to store and access credentials securely, and then use them to connect to data sources. For more information, see Using secrets from vaults in connections.

Metadata import from additional data sources

Metadata import now supports the following data sources:

• Netezza®
• Salesforce.com

Additional connections are synced

Connections to the following data sources are now also synced between the information assets view and the default catalog or when metadata is synced with external repositories:

• Amazon S3
• Microsoft Azure Data Lake Store
• Microsoft Azure File Storage
• Microsoft Azure Blob Storage
• SAP HANA

For details, see:

• Information assets view
• Automatically synchronizing catalog assets and governance artifacts with an external repository

Custom category roles

In addition to the predefined category roles, you can create custom roles with a custom set of permissions. Custom category roles offer more granular control over the actions that users can take within a category. For details, see Category collaborator roles.

Enhancements to importing and exporting governance artifacts using ZIP files

• The relationship between artifacts are defined by the artifact identifiers. This method replaces the previous method, which used the context and name of the artifact to define the relationship. This change ensures that the artifacts are identified consistently.
• The Manage glossary permission was added to control who can import or export governance artifacts using a ZIP file.
• If the synchronization phase of the import process is interrupted, for example because a pod stops running, Watson Knowledge Catalog starts a new pod to restart the synchronization process.

For details, see Importing all governance artifacts from a ZIP file.

Extended support for custom attributes

In previous refreshes, you could apply custom attributes to governance artifacts. However, you could not apply custom attributes to classifications or reference data.

In this refresh, you can now apply custom attributes to classifications or reference data by using the Watson Data REST APIs. For details, see Custom attributes for assets and artifacts.

Give users temporary or role-based access to your Amazon S3 data

The Amazon S3 account owner can provide temporary security credentials or grant role-based access to trusted users for data that is accessed through an Amazon S3 connection. This feature provides greater security and flexibility because the account owner does not need to add additional users to their IAM account. For details, see Setting up temporary credentials or a Role ARN for Amazon S3.

Support for new connection type

Watson Knowledge Catalog can now connect to Databases for DataStax.

Duplicate asset handling

You can now specify how to handle duplicate assets in a catalog:

Update

Update the values of the original assets with the values of the new assets. If the new assets have empty values, the corresponding values from the original assets are retained.

The privacy setting, asset owner, asset members, and activities of the asset remain unchanged.

Duplicate

Add the new assets as duplicates of the original assets. (This is the default behavior.)

Overwrite

Overwrite all values of the original assets with the values of the new assets.

The privacy setting, asset owner, asset members, and activities of the asset remain unchanged.

Reject and preserve

Reject the new duplicate assets and preserve the original assets.

You can change the duplicate asset handling behavior at any time from the catalog Settings page. For details, see Changing catalog settings.

Support for Power® 10 hardware

You can now install Watson Knowledge Catalog on Red Hat® OpenShift® Container Platform Version 4.8 clusters running on Power 10 hardware. However, the service does not take advantage of Power 10 optimizations.

Bug fixes

Catalog: asset preview

Issue: Preview will not work for synced SAP HANA table assets that are contained in a package.

Resolution: Data preview will work on synced SAP HANA assets for tables that are contained in a package.

Issue: PDF preview is no longer available after switching tabs.

Resolution: PDF preview is now viewable after switching tabs.

Issue: Error occurs when previewing data assets from cloud object storage.

Resolution: Preview of data assets from cloud object storage now loads as anticipated.

Catalog: profiling usability

Issue: Automatic profiling is not triggered when syncing data assets from IGC to the default catalog.

Resolution: Assets will be automatically synced to the default catalog.

Catalog: usability

Issue: You cannot preview connected assets where the connection path contains the dot character '.'.

Resolution: The file type and data format are now determined in a different way, so that assets of supported file types can be previewed even if the path contains a dot.

Issue: Large number of concurrent notifications causes 504 timeout responses.

Resolution: All notifications will now be received without causing 504 timeout responses.

Issue: data_definition is registered automatically as a global asset.

Resolution: data_definition is not registered automatically as a global asset.

Issue: Preview of Azure Data Lake data assets synced from IGC does not work correctly and displays incorrect data.

Resolution: The proper ADLS asset columns are now displayed as expected and with the proper number of sampled rows.

Issue: When an asset that has a term assigned is added to a project and republished to a catalog under a different name, it is not listed correctly in the term's related content view. Expanding column information for the related catalog asset results in an error.

Resolution: The details of related catalog assets can now be viewed.

Data discovery: usability

Issue: Quick scan discovery error occurs when a hyphen appears in table name

Resolution: Special characters will not trigger failure if user selects schema (individual table with hyphen still fails).

Issue: Quick scan data assets statuses reset to 'Submitted' after publishing failure.

Resolution: Publish job failed now displays a 'Loading Error' status for quick scan data the assets.

Issue: If a quick scan job runs for over 13 hours, the job might fail due to the token expiry resulting in data class retrieval failure.

Resolution: The token now regenerates and is used if it expires while the quick scan job is still in progress.

Issue: Quick scan results fail to publish to a catalog with an Oracle SSL connection.

Resolution: Quick scan results are now published successfully with an Oracle SSL connection.

Issue: In the data quality data type properties view, when "VarChar(0)" is selected, incorrect data type violation results and quality scores are displayed when the column analysis and data quality analysis are re-run.

Resolution: The user is now prevented from selecting "VarChar(0)" as the selected data type.

Issue: When a Hive SSL platform connection that was already added to data discovery is modified, quick scan jobs for that connection might fail.

Resolution: Quick scan jobs now run successfully if the Hive SSL platform connection which was already added to data discovery is modified.

Data quality: usability

Issue: It's not possible to see the data rule execution history details in a data quality project that was created before the upgrade. Error message CDICO0100E: Connection failed: Sql non transient connection error: IAUSER ;QUIESCE RESTRICTED ACCESS is issued.

Resolution: Data rule execution history details in a data quality project are now present after the upgrade.

Issue:Column analysis and data quality jobs run on a JSON file produces an error and fails.

Resolution: Column analysis and data quality jobs now run successfully on a JSON file.

Issue:Quick scan cannot run without both the Manage asset discovery and Manage data quality permissions.

Resolution: The Manage data quality permission is no longer required to run a quick scan.

Issue: You can't create a rule set by clicking the Create rule set button on the Data rules tab in a data quality project.

Resolution: You can now create a rule set from a rule set definition by using the Create rule set button.

Issue: Data quality details do not refresh when you check the data asset details.

Resolution: Data quality details now refresh when you check the data asset details.

Globalization

Issue: When accessing the Information Assets view in a non-English locale, the Hierarchies tab shows additional entries.

Resolution: The list of hierarchies is now the same in all locales.

Issue: When using workflow management, or access control in role listing, in a non-English locale, the Watson Knowledge Catalog platform permissions are displayed as unreadable strings.

Resolution: Watson Knowledge Catalog platform permissions now translate correctly in the non-English locale.

Governance artifacts: usability

Issue: Custom relationships for artifacts other than business terms can't be deleted.

Resolution: Custom relationships can now be deleted for all types of governance artifacts.

Issue: ZIP export fails if user lacks minimum permissions in any of the categories.

Resolution: ZIP export now completes successfully.

Issue: In reverse custom relationships, the link to the source artifact is broken.

Resolution: The links in reverse custom relationships now work.

Issue: Errors occur when adding or editing custom relationships for governance rules.

Resolution: Adding or editing custom relationships for governance rules will not produces errors.

Issue: Importing over 1,000 classifications to the glossary takes six or more hours to complete.

Resolution: Importing large classification sets now complete within minutes.

Issue: Permissions defined on categories are ignored when importing glossary artifacts from a ZIP file to existing categories.

Resolution: You must now have the "Glossary administrator" permission to import from ZIP.

Issue: Data Refinery flows that access data assets where data is masked fail on Power.

Resolution: Data Refinery flows now work on Power when data protection rules are applied.

Issue: User is not able to create an artifact for a primary category to which he has an editor permission.

Resolution: User can create artifacts for categories where he has an editor permission.

Governance artifacts: IGC Migration

Issue: Unable to assign Cloud Pak for Data users as data stewards.

Resolution: The CPD user is now successfully synced as a steward for information assets when the user is assigned the CPD Data Steward role.

Metadata import: usability

Issue: Parquet files imported using metadata import can’t be profiled because they have the mime type application/x-parquet.

Resolution: Profiling now supports the mime type application/x-parquet.

Profiling: usability

Issue: Data types are not shown in profiling results for DV assets.

Resolution: The schema details are now loaded in a different way, so that the data types are included in the profiling results for DV assets.

Security fixes

This release includes fixes for the following security issues:

CVE-2014-6393

CVE-2016-5017

CVE-2017-5637

CVE-2018-1109, CVE-2018-16487, CVE-2018-19057, CVE-2018-3721, CVE-2018-3767, CVE-2018-8012

CVE-2019-1010266, CVE-2019-10744, CVE-2019-3881

CVE-2020-12265, CVE-2020-14039, CVE-2020-14330, CVE-2020-15168, CVE-2020-24025, CVE-2020-25648, CVE-2020-26160, CVE-2020-28469, CVE-2020-28493, CVE-2020-28500, CVE-2020-35492, CVE-2020-36048, CVE-2020-36327, CVE-2020-8175, CVE-2020-8203, CVE-2020-9484

CVE-2021-20066, CVE-2021-20191, CVE-2021-20277, CVE-2021-22922, CVE-2021-22923, CVE-2021-22924, CVE-2021-23337, CVE-2021-23343, CVE-2021-23413, CVE-2021-23424, CVE-2021-23436, CVE-2021-23440, CVE-2021-25122, CVE-2021-25329, CVE-2021-27218, CVE-2021-30640, CVE-2021-31799, CVE-2021-31810, CVE-2021-32066, CVE-2021-32803, CVE-2021-32804, CVE-2021-33037, CVE-2021-33623, CVE-2021-33813, CVE-2021-3583, CVE-2021-36159, CVE-2021-36222, CVE-2021-3664, CVE-2021-3690, CVE-2021-3711, CVE-2021-3749, CVE-2021-3757, CVE-2021-37701, CVE-2021-37712, CVE-2021-37713, CVE-2021-37714, CVE-2021-37750, CVE-2021-3777, CVE-2021-3801, CVE-2021-3803, CVE-2021-3828, CVE-2021-40330, CVE-2021-41079, CVE-2021-41617

Refresh 2 of Cloud Pak for Data Version 4.0

A new version of Watson Knowledge Catalog was released in October 2021.

Operand version: 4.0.2

This release includes the following changes:

New features

The 4.0.2 release of Watson Knowledge Catalog includes the following features and updates:

Support for Power

You can install Watson Knowledge Catalog on a Red Hat OpenShift Container Platform Version 4.8 cluster running on Power hardware.

Data discovery from Netezza sources

You can run automated discovery and quick scan jobs on Netezza data source by using a generic JDBC platform connection. For information on creating platform connections, see Connecting to data sources at the platform level.

Usability improvement for editing governance artifacts

When you edit a governance artifact property and select an artifact, you can display basic information for the selected artifact in the same edit panel.

Support for new connection type

Watson Knowledge Catalog can now connect to Amazon RDS for Oracle.

Support for additional data sources

Metadata import now supports the following data sources:

• Databases for MongoDB
• MongoDB
• SAP HANA

Bug fixes

Catalog: asset preview

Issue: The download option is not available for assets that don't support preview.

Resolution: The download button is now enabled for assets that don't support preview.

Catalog: profiling usability

Issue: Profiling results might show invalid negative values for matches, mismatches, and missing.

Resolution: The download action now works consistently when the download button is clicked.

Catalog: usability

Issue: Downloading a data asset in a catalog fails after switching between tabs within a single asset.

Resolution: The download action now works consistently when the download button is clicked.

Custom workflows: usability

Issue: When the wkc-workflow-service pod is configured to use a time zone other than UTC, requests to the Workflow API will fail and no governance artifacts can be created.

Resolution: The wkc-workflow-service pod is no longer required to run with the UTC time zone (although it is not recommended to change the time zone).

Data discovery: connections

Issue: When you run a discovery job on a Snowflake data source, row limits might not be honored.

Resolution: Row limits are now properly honored when you run discovery jobs on Snowflake data sources.

Data discovery: performance

Issue: Publishing data assets from a large number of data assets in the quick scan results takes a long time because information for all the data sets is retrieved together.

Resolution: Information of the specific data assets to publish is now retrieved individually for each data asset.

Data discovery: usability

Issue: Automated discovery might produce suboptimal or incorrect term assignments.

Resolution: Automated discovery term assignments have been improved in the following ways:

• Improved matching when there are multiple data class matches
• Improved name matching terms
• Ability to specify different strategies when computing the final term confidence

Issue: Custom scaling of the odf-fast-analyzer pod is not possible.

Resolution: To scale the odf-fast-analyzer pod, you can now set the variable odf_fast_analyzer_replicas in the Unified Governance custom resource to the wanted number of replicas.

Issue: Quick scan results show only data sets that were successfully analyzed, but do not include the data sets that failed discovery.

Resolution: Quick scan results now show both the successfully analyzed data sets and the data sets which failed discovery.

Issue: NaN or infinite values in tables cause errors in automated discovery.

Resolution: Automated discovery now completes successfully if tables contain NaN or infinite values in the table.

Data quality: performance

Issue: The performance of the Data Quality UI is poor after a restart of services or pods when there are thousands of assets with many term assignments.

Resolution: Startup for the term assignment service is now optimized to avoid unnecessary processing when the service is started.

Data quality: scalability

Issue: Column scrolling is slow when you view the data quality details for a table with thousands of columns and might lead to a UI crash.

Resolution: The scrolling capability was improved to better handle the scrolling of thousands of columns.

Data quality: usability

Issue: Data type percentage values and data class matching values are not included when data quality details are exported to the file.

Resolution: The exported file now includes data type percentage values and data class matching values.

Issue: In the data assets view of data quality projects, the total number of data assets and the number of reviewed data assets are not updated when the refresh button is clicked.

Resolution: The numbers are now updated when refresh is clicked after adding data assets to the data quality project.

Issue: Some tables might show an incorrect data quality score of 0% in the tile view while showing the correct data quality score in the list view.

Resolution: The list view and tile view now show the correct data quality score.

Governance artifacts: Information Governance Catalog migration

Issue: After you upgrade from 3.5.x to 4.0.1, the InfoSphere® Information Server glossary migration service is unavailable.

Resolution: The glossary migration service is now automatically restarted after the upgrade and is available to users.

Issue: Data classes of the type OtherMatchingCriteria imported in the Information Governance Catalog are not migrated to Watson Knowledge Catalog.

Resolution: Data classes of the type OtherMatchingCriteria can be successfully imported and migrated to Watson Knowledge Catalog.

Governance artifacts: usability

Issue: In a Chrome browser, business terms with longer descriptions have overlapping text.

Resolution: Longer business term descriptions are now displayed properly.

Issue: Reverse custom relationships are not shown in the target governance artifact when the source and target of the relationship are of different artifact types.

Resolution: API and UI have been updated to display reverse custom relationships in the target governance artifact when the source and target are of different artifact types.

Issue: Importing a file that contains updates to relationships for existing business terms might delete those business terms.

Resolution: The file import updates to the business term relationships are now applied successfully to the business terms without deleting them.

Issue: After deleting a draft created for a published artifact it is not possible to edit that published artifact again.

Resolution: The published artifact can now be modified successfully after deleting a draft of it.

Issue: In some cases, secondary categories might not be displayed in the context path of business terms.

Resolution: The secondary category now displays properly in the context path.

Issue: Custom attributes created after the upgrade to Cloud Pak for Data 4.0.1 might not be successfully exported.

Resolution: Custom attributes can now be successfully exported.

Issue: If you create a custom asset with a mixed-case name, you will not be able to edit the custom attribute values.

Resolution: You can now successfully edit the custom attributes of a custom asset type with a mixed-case name.

Issue: When you change the matching method for a data class, the matching criteria is not updated.

Resolution: The matching criteria is now properly updated when you change the matching method for a data class.

Governance workflows

Issue: The due date of a workflow task is saved in UTC time format. This can lead to a different due date shown in the task inbox in some time zones.

Resolution: The correct due date is now shown.

Issue: In the task inbox, a comment that is added to, edited, or deleted from the activity panel is not visible until the panel is refreshed manually.

Resolution: The activity panel now refreshes automatically after a comment is added.

Issue: The task inbox continues to show a task after an action is completed with the action button remaining visible. An error occurs when the task is clicked.

Resolution: Clicking the task's action button after completion of the task no longer results in an error.

Issue: Clicking a task link from an email notification does not show the task in the task inbox.

Resolution: The appropriate task is now displayed in the task inbox when clicking that task link from an email notification.

Issue: In the task inbox, if a custom workflow request task is at the beginning of the list of assigned tasks, subsequent tasks in the list—such as import, approve, or publish—can not be completed until the first task is completed.

Resolution: Subsequent tasks can now successfully be completed first when there is a custom workflow task higher up in the task inbox.

Security fixes

This release includes fixes for the following security issues:

CVE-2017-14502

CVE-2020-25658, CVE-2020-36323

CVE-2021-22918, CVE-2021-23017, CVE-2021-23358, CVE-2021-23362, CVE-2021-2341, CVE-2021-2369, CVE-2021-2388, CVE-2021-27290, CVE-2021-28875, CVE-2021-28876, CVE-2021-28877, CVE-2021-28878, CVE-2021-28879, CVE-2021-31162, CVE-2021-35065

Refresh 1 of Cloud Pak for Data Version 4.0

A new version of Watson Knowledge Catalog was released in August 2021.

Operand version: 4.0.1

This release includes the following changes:

New features

The 4.0.1 release of Watson Knowledge Catalog includes the following features and updates:

Support for upgrade

You can now upgrade Watson Knowledge Catalog from the following Cloud Pak for Data releases:

• Cloud Pak for Data Version 3.5.x
• Cloud Pak for Data Version 4.0.x

Data discovery: quick scan results

This release includes the following changes to data discovery:

• The View data quality permission now grants access to quick scan results.
• You can reanalyze individual tables.
• You can set the status Reviewed for columns.
• You can enable overwrite of existing term assignments when results are republished.

For details, see: Reviewing and working with quick scan results

Data discovery: automated discovery

You can now select multiple folders and individual files for discovery for file connection types such as HDFS. For details, see Running automated discovery.

Export and import of all governance artifacts from a single file

You can now export all governance artifacts to a single ZIP file and import them all at once by using REST API. For details, see:

• Exporting all governance artifacts to a ZIP file
• Importing all governance artifacts from a ZIP file

Metadata sync with external repositories

You can configure Watson Knowledge Catalog to sync governance artifacts and catalog assets with external repositories such as other instances of Watson Knowledge Catalog, IBM InfoSphere Information Governance Catalog, or Apache Atlas. The external metadata repositories must comply with ODPi Egeria standards and support its Open Metadata Repository Services (OMRS). Synchronization between the repositories happens through participation in an Egeria cohort.

For details, see: Synchronizing with an external repository

Tech preview This is a technology preview and is not supported for use in production environments.

Improved performance in governance workflows

The performance was improved for workflow tasks with larger sets of data, for example when you import governance artifacts.

Support for new connection type

Watson Knowledge Catalog can now connect to SQL Query.

Support for additional data sources

Metadata import now supports the following data sources:

• MariaDB
• Snowflake
• SQL Query

Support for tags in categories

You can now assign one or more tags to a category.

For details, see: Managing categories

Bug fixes

Catalog: asset sync

Issue: The relationship between data class and business term is not synced from the Information Assets view to Watson Knowledge Catalog. 

Resolution: During data discovery if an assigned data class has an associated business term, that business term shows as a suggested term in the discovery results.

Issue: After a term is removed from a non-primary category (for the particular term) in Watson Knowledge Catalog, that relationship is still being displayed in the Information Assets view.

Resolution: Removal of the non-primary category relationship is now synced properly to the Information Assets view.

Issue: When you edit a category hierarchy in the Information Assets view, those changes are not synced to the Watson Knowledge Catalog default catalog.

Resolution: Setting of parent-child category relationships in a category in the Information Assets view now syncs to the Watson Knowledge Catalog default catalog.

Issue: When you update governance artifact properties in the Information Assets view, the changes are not synced to the Watson Knowledge Catalog default catalog.

Resolution: Updates of governance artifact properties, such as editing a business term description, now sync properly to the Watson Knowledge Catalog default catalog.

Issue: If bidirectional syncing of governance artifacts is enabled, deleting a relationship between artifacts does not sync to the Watson Knowledge Catalog default catalog.

Resolution: Deleting a relationship between artifacts now syncs to the Watson Knowledge Catalog default catalog.

Issue: Connected assets are deleted from the Information Assets view when the connection in the default catalog is deleted.

Resolution: The connected assets now remain in the Information Assets view when their associated connection is deleted in the default catalog.

Catalog: governance artifacts

Issue: When you add catalog assets from a local file and are assigning classifications, you cannot scroll past the first 100 classifications.

Resolution: The scroll behavior now works correctly and you can view all available classifications.

Catalog: profiling

Issue: For some browser resolution settings, the horizontal and vertical scroll bars on the profile tab flicker. This problem happens for Mozilla Firefox and for Google Chrome web browsers.

Resolution: This behavior is corrected and no longer occurs in the specified browsers.

Issue: Updating the profile fails for the user who does not provide personal credentials for connection that is created by another user.

Resolution: If the user who is updating the profile did not create the personal connection, a dialog prompt is displayed to enter credentials for the connection to be able to update the profile.

Catalog: usability

Issue: The catalog UI does not load when an asset has an assigned term that is missing its display name.

Resolution: The catalog UI now displays a message that states that the assigned term for the asset is missing its display name.

Issue: When you configure data class matching, a valid regular expression might fail validation in the UI.

Resolution: UI now successfully validates valid regular expressions that were failing before.

Data Curation: auto discovery

Issue: With hierarchical data classes such as the Driver's license data classes, some false positive data classifications are identified for columns.

Resolution: Make sure that the parent data class and the child data class are a match to prevent these false positive scenarios.

Issue: Data classification results are incorrect if Column Analysis is not specified when you run auto discovery.

Resolution: Data classification now produces correct results if Column Analysis is not specified when you run auto discovery.

Issue: You are unable to run auto discovery for a Db2 connection with the SSL option enabled.

Resolution: Auto discovery now runs successfully for a Db2 connection with the SSL option enabled.

Data Curation: quick scan discovery

Issue: Newly created data classes are not picked up by subsequent quick scan analysis on existing data quality projects.

Resolution: Subsequent quick scan analysis will now pick up and apply the classifications for data classes that are created after the previous quick scan analysis job.

Issue: If the name of the data quality project that is selected for a quick scan job contains spaces or non-ASCII characters, the data classes that are defined in that data quality project cannot be applied. Instead, the default data classes are used.

Resolution: Quick scan discovery now loads and applies the project data classes if the project has spaces in its name.

Issue: Newly created data classes are not used by quick scan.

Resolution: Quick scan checks for newly created data classes before discovery jobs are run.

Issue: Publish of quick scan results is not synced to the Information Assets view when the user who does the publish, and has the editor role for the default catalog and the viewer role for the platform assets catalog, did not create the connection that is used for the quick scan discovery.

Resolution: The user who did not create the connection but has the proper roles is able to successfully publish the assets and connection, and the results are synced and are available in the Information Assets view.

Issue: Quick scan discovery does not work with Japanese schema name.

Resolution: Quick scan discovery now works with Japanese schema name and table names.

Issue: Republishing a discovered table to the default catalog after you drop a column is not syncing the column change to the Information Assets view.

Resolution: Now the sync of the republished quick scan results properly deletes the dropped column in the Information Assets view.

Data Curation: resiliency

Issue: Quick scan delta analysis discovery fails if the column name contains double quotation marks.

Resolution:Quick scan delta analysis now runs successfully on columns where the column name contains double quotation marks.

Data Curation: security roles

Issue: User with the Manage data quality permission is not able to view or work with automation rules.

Resolution:User with the Manage data quality permission can now view or work with automation rules.

Data Curation: usability

Issue: When you are assigning terms, no notification appears that term assignment is in progress, and after completion no notification appears that term assignment was completed successfully.

Resolution: A message displays stating that update of term assignments is in progress, and after term assignment is complete the table details is automatically refreshed.

Issue: In a data quality project, when you switch across columns in a data asset, the term assignment on the columns tab is cleared.

Resolution: The term assignments are displayed correctly in the columns tab without having to manually refresh the view.

Issue: When you publish quick scan results for assets, where the associated data connection no longer exists, it shows a Loading Error status but the message that appears does not explain why the publish failed.

Resolution: When you publish the results, a warning appears in the publish dialog that states that the test of the connection failed. If the results are still published and show the Loading Error status, the messages state that the connection was not found and added to the catalog.

Issue: If your attempt to add a connection with an SSL certification to data discovery fails (for various reasons), the subsequent attempt to add the connection might fail because a certificate alias with the name might exist from the previous failed attempt.

Resolution: Logic was added so that if the connection with an SSL certificate is not successfully added to data discovery, the certificate is cleaned up to prevent certificate alias exists scenario.

Issue: You are unable to view the frequency distribution drill-down data on published data quality assets that are added to a new data quality project if those assets are published from quick scan results.

Resolution: For assets that are published from quick scan results, the frequency distribution drill-down data now displays correctly after you run a column analysis on those assets.

Data Quality: data rules

Issue: When you configure data rule bindings and table join conditions, you are not able to see the associated schema for the associated tables for selected columns.

Resolution: A tooltip is now provided so that you can see the schema details for the selected columns when you configure bindings and join conditions.

Data Quality: scalability

Issue: Automation rules fail to work if a column analysis job contains thousands of columns.

Resolution: Automation continues to work as expected as the number of columns in the column analysis are scaled to up to many thousands.

Data Quality: usability

Issue: You might not able to scroll all data assets in a data quality project in the tile view under certain conditions.

Resolution: Scroll of data assets in a data quality project in the tile view now works in all supported browsers and at different zoom views.

General: user roles

Issue: "Create Catalog" permission does not have any effect when applied.

Resolution: The permission was removed, as the capability intended as the "Manage catalogs" permission can be used in its place.

Governance: artifact workflows

Issue: For a task in the task inbox, the result is displayed instead of the artifact type, and workflow status.

Resolution: For single artifact tasks, the proper task and status are now displayed in the task inbox.

Governance artifacts: custom workflows

Issue: While in the task inbox, if the custom workflow request task is at the beginning of the assigned task list, the subsequent tasks, such as Import, Approve, or Publish, cannot be completed until the beginning task is completed.

Resolution: The tasks in the task inbox can now be completed without having to complete the task at the beginning of the task inbox list.

Governance artifacts: performance

Issue: Import of 1000 governance artifacts might fail with a timeout exception.

Resolution: Optimized import of governance artifacts to prevent the timeout scenario.

Governance artifacts: usability

Issue: When you edit a reverse custom relationship, more entries are added than expected.

Resolution: When you edit a reverse custom relationship, the changes and entries are added as expected.

Issue: Custom relationships for categories are visible in UI only if a regular custom attribute is defined for categories.

Resolution: Custom relationships for categories are now displayed even if no custom attributes are defined.

Issue: Existing custom columns are not retrieved when you try to add custom columns while you are creating a reference data set.

Resolution: Existing custom columns are retrieved and displayed properly in the New reference data set catalog.

Issue: In the New data protection rule dialog, the auto-complete search for business terms does not return wanted terms based on filter criteria due to a limitation on the length of results that are returned.

Resolution: The auto-complete logic was changed to return the most relevant results rather than a sorted list, which might have too many matches to display or be useful.

Issue: An error is produced when you try to unassign related artifacts of a business term and the artifact is not unassigned.

Resolution: Artifacts are now successfully unassigned when the artifacts relationships are deleted.

Issue: It is not possible to have two different custom relationships between the same governance artifacts.

Resolution: You can now set multiple custom relationships between the same governance artifacts.

Issue: You cannot add multiple custom relations to the same artifact type.

Resolution: You can now add multiple custom relations to the same artifact type successfully.

Issue: When a custom relationship in a governance artifact has more than five values, only five are visible.

Resolution: Added pagination to show more than five custom relationships for the business term in the term details view.

Governance workflow: usability

Issue: When you upload an asset for a business term and clicking the link to the task inbox, the task inbox does not display the task.

Resolution: The associated task is now displayed in the task inbox when you click the link to go to the task inbox.

Metadata import: usability

Issue: The "Created by" and "Modified" fields always show "Not Applicable" when you view metadata imports in an analytics project.

Resolution: Accordingly, "Created" by and "Modified by" details are now displayed when you view metadata imports details.

Platform connections

Issue: Importing a Microsoft Excel file from a Box data source fails if the Excel file contains one or more empty sheets.

Resolution: The blank sheet is now imported successfully and can preview the other imported sheets.

Platform connections: Apache Hive connector

Issue: User cannot configure an Apache Hive configuration by using Kerberos authentication.

Resolution: User can now specify a Kerberos keytab file for the Apache Hive connector.

Platform connections: Excel connector

Issue: When you view the output of an Excel sheet, the rows are not displayed correctly when First line is header checkbox is not selected.

Resolution: Output of the Excel sheet is now displayed correctly when the First line is header checkbox is not selected.

Platform connections: HDFS data source

Issue: Data refinery job fails when you are modifying a connected asset that is created from an HDFS connection with Use home as root checked.

Resolution: Data refinery job now completes as expected with the HDFS connection with Use home as root checked.

Security fixes

This release includes fixes for the following security issues:

CVE-2016-10228

CVE-2018-1109, CVE-2018-7489

CVE-2019-10241, CVE-2019-12402, CVE-2019-25013, CVE-2019-9169

CVE-2020-10673, CVE-2020-15945, CVE-2020-25692, CVE-2020-27618, CVE-2020-28469, CVE-2020-28491, CVE-2020-28493, CVE-2020-36048, CVE-2020-7753, CVE-2020-7768

CVE-2021-20191, CVE-2021-23343, CVE-2021-23358, CVE-2021-27219, CVE-2021-29505, CVE-2021-30468, CVE-2021-32640, CVE-2021-3326, CVE-2021-33502, CVE-2021-33503

Initial release of Cloud Pak for Data Version 4.0

A new version of Watson Knowledge Catalog was released as part of Cloud Pak for Data Version 4.0.

Operand version: 4.0.0

This release includes the following changes:

New features

Version 4.0.0 of the Watson Knowledge Catalog service includes the following features and updates:

Enhancements when importing COBOL copybooks

When you import COBOL copybooks, the relationships between the copybooks and the corresponding virtual tables are imported into the catalog.

You can also select an individual COBOL copybooks for metadata import.

In addition, the performance of importing COBOL copybook metadata is improved.

For details, see Importing metadata.

Use groups to manage data governance

You can use groups to add collaborators to:

• Categories

  For details, see Managing access to a category.

• Catalogs

  For details, see Managing access to a catalog.

You can also specify user groups in:

• The condition of a data protection rule

  For details, see Managing data protection rules.

• The workflow configurations for governance artifacts

  For details, see Configuring custom workflows.

User groups are currently not supported in data quality projects.

Support for new connection types

Watson Knowledge Catalog can now connect to:

• Databases for MongoDB
• Microsoft Azure File Storage

In addition, the following connection names have changed:

• Sybase is now SAP ASE
• Sybase IQ is SAP IQ

This change impacts only the connection type names. The connection settings remain the same.

Usability improvements for metadata import

Metadata import includes support for Box as a data source. It also includes the following improvements:

• Create and add tags to the metadata import asset
• Directly edit the configuration from the review section
• Edit a metadata import asset from within the asset
• See the status of imported data assets
• More options when setting the data scope

For details, see Importing metadata.

Assign custom attributes to default asset types

The default asset types that are included with Watson Knowledge Catalog can now have custom attributes. Because a default asset type cannot be directly modified, you use the API to apply custom attributes from one or more other asset types to the default asset type, which gives the default asset type custom attributes.

For details, see Adding assets to a catalog

Custom relationships between governance artifacts

You can now create and use custom attributes of the type relationship to define relationships between governance artifacts.

For details, see Custom attributes.

Data protection rules enhancements

You can now use column names in rule conditions, and you can mask columns based on the business terms, data classes, or tags assigned to a column or based on the column name.

For details, see Managing data protection rules.

Data discovery and data quality enhancements

Additional connections

You can now use the following connection types in quick scan, automated discovery, and data quality projects:

• Amazon Redshift
• Apache Kudu
• Data Virtualization

For details, see Discovering assets.

More details in the run history of data rule sets

The run details of a data rule set now include an Output tab where you can see the output data for the configured output setting.

For details, see Running rule sets.

Quick scan results UI

You can now do bulk assignments or removals of business terms, and you can publish results at schema level.

For details, see Working with quick scan results.

Support for audit logging

Watson Knowledge Catalog integrates with the Cloud Pak for Data audit logging feature. Events in the following areas generate logs:

• Metadata import
• Policies
• Policy rules
• Profiling
• Catalog
• Workflow
• Business glossary

For details, see Services that support logging.

Smaller installation footprint

You can optionally install the Watson Knowledge Catalog service without the legacy user interface that provides advanced curation and data quality.

You can choose between the core installation or the full installation. For details, see Installing Watson Knowledge Catalog.

Improved search across the platform

You can now use the global search bar to search for assets across all the projects, catalogs, and deployment spaces to which you have access. You can also search for governance artifacts across the categories to which you have access.

The search now finds results across more asset properties and governance artifacts. You can now search for exact words or phrases by surrounding search terms with double quotation marks. For details, see Searching across the platform.