What's new and changed in Watson Query
Watson Query updates can include new features, bug fixes, and security updates. Updates are listed in reverse chronological order so that the latest release is at the beginning of the topic.
You can see a list of the new features for the platform and all of the services at What's new in IBM Cloud Pak® for Data.
Installing or upgrading Watson Query
Ready to install or upgrade Watson Query?
- Related documentation:
Cloud Pak for Data Version 4.7.3
A new version of Watson Query was released in September 2023 with Cloud Pak for Data 4.7.3.
Operand version: 2.1.3
This release includes the following changes.
- New features
-
The 2.1.3 release of Watson Query includes the following features and updates:
- Log audit events when Watson Query is deployed in a tethered namespace
- You can now log audit events when an instance is provisioned in an OpenShift® project separate from Cloud Pak for Data (or a "tethered project"). For more information on audit events, see Audit events for Watson Query.
- Administrators can now make virtual objects visible to all users
- Administrators can now choose to give users a more comprehensive view of the content by making existing virtual objects visible from the Virtualized data page. Data access within those objects continues to adhere to Watson Query authorizations and data protection rules. To enable this feature, administrators need to disable the Restrict visibility setting from Service settings. For more information, see Managing visibility of virtual objects in Watson Query.
- Easily format how EXPLAIN information appears for query access plans
- You can now use the Cloud Pak for Data web client to
format how EXPLAIN information appears when you generate query access plans. You can then run the
db2exfmt
command from the web client to easily generate and download the EXPLAIN output in text files. For more information about thedb2exfmt
command, see db2exfmt - Explain table format command in the Db2® documentation. - Use wildcard characters to filter your data sources
- Now when you create a virtualized table, you can use the following wildcard characters:
- % (percent): To represent zero or more characters
- _ (underscore): To represent a single character
You can use the wildcard characters to customize filters to find the data sources that you need. For more information, see Filtering data in Watson Query.
- Configure PEP cache settings from the web client
- You can now use the Cloud Pak for Data web client to configure policy enforcement point (PEP) cache settings, such as cache size and cache live time, for data protection rules. For more information on PEP caches, see Enabling enforcement of data protection rules in Watson Query.
- Watson Query users can publish their own virtual objects
- Users with the
User
role in Watson Query can now publish virtual objects that they created to governed catalogs. For more information, see Publishing virtual data to a catalog with Watson Query.
- Customer-reported issues fixed in this release
- Security fixes
-
This release includes fixes for the following security issues:
CVE-2023-1281, CVE-2023-1667, CVE-2023-1829, CVE-2023-2124, CVE-2023-2194, CVE-2023-2235, CVE-2023-2283, CVE-2023-2602, CVE-2023-2603, CVE-2023-2727, CVE-2023-2728, CVE-2023-2828, CVE-2023-3347, CVE-2023-3899, CVE-2023-22045, CVE-2023-22049, CVE-2023-24329, CVE-2023-24532, CVE-2023-24534, CVE-2023-24536, CVE-2023-24537, CVE-2023-24538, CVE-2023-26604, CVE-2023-27535, CVE-2023-27536, CVE-2023-28321, CVE-2023-28370, CVE-2023-28466, CVE-2023-28484, CVE-2023-29469, CVE-2023-32002, CVE-2023-32006, CVE-2023-32360, CVE-2023-32559, CVE-2023-32681, CVE-2023-34969, CVE-2023-38408, CVE-2023-40590
CVE-2022-35252, CVE-2022-36227, CVE-2022-41724, CVE-2022-41725, CVE-2022-42896, CVE-2022-43552, CVE-2022-45061, CVE-2022-46751, CVE-2022-48281
CVE-2020-24736, CVE-2020-25649
Cloud Pak for Data Version 4.7.0
A new version of Watson Query was released in June 2023 with Cloud Pak for Data 4.7.0.
Operand version: 2.1.0
This release includes the following changes.
- New features
-
Version 2.1.0 of the Watson Query service includes the following features and updates.
- Choose your query mode to prioritize either performance or consistency
- You can now choose between running queries in Max Pushdown mode or in Max Consistency mode.
- Max Pushdown mode ignores semantic difference between Watson Query and data source for single source queries. Therefore, more single source queries might be fully pushed down to data source, improving query performance. Query results are consistent with data source semantics for fully pushed down queries in this mode. Max Pushdown mode does not impact mulitple-source queries.
- Max Consistency mode follows Watson Query semantics to evaluate whether operations can be pushed down to the data source. If the operation that is executed on the data source generates the same result as Watson Query, the operation can be pushed down. Queries in this mode might be fully pushed down if the remote data source has the same semantics as Watson Query.
- Pushdown enhancements to improve query performance
- Query pushdown is an optimization feature that reduces query times and memory use. This release
of Watson Query includes the following
enhancements in queries that use pushdown:
- The following data source connections have been optimized to take advantage of more data source
capabilities to improve query performance on single-source tables:
- Salesforce.com
- Db2 for i
- Query performance is improved in pushdown mode in the following situations:
- When you query string data on remote data sources with the IN predicate. For details about the IN predicate, see IN predicate in the Db2 documentation.
- When you query data where the total width of the columns in the Select list is greater than 32 thousand.
- When you use common sub-expressions (CSE) pushdown capabilities.
- When you reference numeric data type functions in the query.
- When you reference date and time type functions in the query.
- The following data source connections have been optimized to take advantage of more data source
capabilities to improve query performance on single-source tables:
- Use your platform credentials to access Watson Query connections
- When you use a platform connection to access Watson Query, you are prompted for your credentials. You can optionally select Use my platform login credentials, rather than entering your personal credentials for the connection. The connection uses your current session JSON Web Token (JWT).
- Use advanced data masking on virtualized data
- In this release of Watson Query, data masking performance is substantially improved. You can now use the advanced data masking options to avoid exposing sensitive data. See Masking virtual data in Watson Query to learn about the updated masking behavior in this release and for instructions on how to revert to the masking behavior from Cloud Pak for Data version 4.6.x if necessary.
- Maintain authorizations when you rename a group
- When you rename a group in Watson Query, you can now migrate the group-level authorizations to the new group name by using the MIGRATE_GROUP_AUTHZ stored procedure. For more information, see MIGRATE_GROUP_AUTHZ stored procedure.
- Connect to data sources that have Kerberos authentication
- You can now connect to data sources that use Kerberos authentication. For more information, see Enabling Kerberos authentication in Watson Query.
- Query data in Microsoft Azure Data Lake Storage Gen2 data lakes
- You can now connect to Microsoft Azure Data Lake Storage Gen2 data sources. For more information, see Supported data sources in Watson Query.
- Manage who can access and perform operations on individual data sources
- With data source access restrictions, you can explicitly manage access to individual data source
connections that use shared credentials. You can assign users, user groups, and roles as
collaborators for a data source connection. Only those collaborators can access the data source
connection. You assign specific privileges to the collaborators to manage the actions that they can
perform on the data sources. This enables you to separate privileges from roles, so that some users
who are assigned a role such as Admin can access and take action on different data source
connections than other Admin users.
For more information, see Data source connection access restrictions in Watson Query.
- Deploy multiple instances of Watson Query
- Previously, you could provision only one Watson Query service instance in a given instance of
Cloud Pak for Data. You can now provision multiple
Watson Query service instances by using
tethered projects.
Each Watson Query service instance must be in a different project. For example, you can provision one service instance to the project where the Cloud Pak for Data control plane is installed, another instance to tethered project A, and a third instance to tethered project B.
- For more information about tethered projects, see Supported project (namespace) configurations.
- For more information on provisioning service instances, see Provisioning the Watson Query service.
- Format and save formatted access plans for performance tuning
- You can now format and save formatted access plans for performance tuning in Watson Query by using the
EXPLAIN_FORMAT
stored procedure. Run this procedure to build query access plans and download the generated EXPLAIN output in text files. For more information, see EXPLAIN_FORMAT stored procedure in Watson Query. - Use improved audit logging to monitor user activity and data access
- You can monitor user activity with additional Watson Query auditable events in the areas of caching and data source isolation. You also now can monitor data access by using the Db2 audit facility. For more information, see Audit events for Watson Query and Auditing in Watson Query.
- Customer-reported issues fixed in this release
-
- DT209719: bigsql/dbstart deployment errors due to INSTANCE_MEMORY
- DT213133: Differentiate between COS Virtualization and Remote Connector File Virtualization in Watson Query in Cloud Pack 4 Data
- DT208441: After OpenShift Container Platform (OCP) and the storage layer upgrade, Watson Query can't start up the QP AdminNode and KeyStore corrupted
- Security fixes
-
This release includes fixes for the following security issues:
CVE-2023-0266, CVE-2023-0361, CVE-2023-0386, CVE-2023-0767, CVE-2023-20860, CVE-2023-20861, CVE-2023-20863, CVE-2023-23916, CVE-2023-25153, CVE-2023-25173, CVE-2023-32233
CVE-2022-2047, CVE-2022-2048, CVE-2022-4269, CVE-2022-4378, CVE-2022-4450, CVE-2022-23471, CVE-2022-25647, CVE-2022-25857, CVE-2022-27191, CVE-2022-27664, CVE-2022-29526, CVE-2022-32149, CVE-2022-38749, CVE-2022-38750, CVE-2022-38751, CVE-2022-38752, CVE-2022-40149, CVE-2022-40152, CVE-2022-41721, CVE-2022-41854, CVE-2022-45688
CVE-2021-22573, CVE-2021-26341, CVE-2021-28165, CVE-2021-28169, CVE-2021-29425, CVE-2021-33813, CVE-2021-34428, CVE-2021-42740
CVE-2020-8840, CVE-2020-9480, CVE-2020-9546, CVE-2020-9547, CVE-2020-9548, CVE-2020-10650, CVE-2020-10672, CVE-2020-10673, CVE-2020-10968, CVE-2020-10969, CVE-2020-11111, CVE-2020-11112, CVE-2020-11113, CVE-2020-11619, CVE-2020-11620, CVE-2020-13936, CVE-2020-13949, CVE-2020-13956, CVE-2020-14060, CVE-2020-14061, CVE-2020-14062, CVE-2020-14195, CVE-2020-15522, CVE-2020-24616, CVE-2020-24750, CVE-2020-26939, CVE-2020-27216, CVE-2020-27218, CVE-2020-27223, CVE-2020-28491, CVE-2020-35490, CVE-2020-35491, CVE-2020-35728, CVE-2020-36179, CVE-2020-36181, CVE-2020-36182, CVE-2020-36184, CVE-2020-36185, CVE-2020-36186, CVE-2020-36187, CVE-2020-36188, CVE-2020-36189
CVE-2019-12086, CVE-2019-12384, CVE-2019-12814, CVE-2019-14379, CVE-2019-14439, CVE-2019-14540, CVE-2019-14892, CVE-2019-14893, CVE-2019-16335, CVE-2019-16942, CVE-2019-16943, CVE-2019-17267, CVE-2019-17531, CVE-2019-20330
CVE-2018-5968, CVE-2018-7489, CVE-2018-12022, CVE-2018-12023, CVE-2018-14718, CVE-2018-14720, CVE-2018-14721, CVE-2018-19360, CVE-2018-19361, CVE-2018-19362
CVE-2017-7525, CVE-2017-15095, CVE-2017-17485, CVE-2018-11307