What's new and changed in Data Virtualization
Data Virtualization updates can include new features and fixes. Releases are listed in reverse chronological order so that the latest release is at the beginning of the topic.
You can see a list of the new features for the platform and all of the services at What's new in IBM Software Hub.
IBM Cloud Pak for Data Version 5.4.0
A new version of Data Virtualization was released in June 2026.
This release includes the following changes:
- New features
-
This release of Data Virtualization includes the following features:
- Data Virtualization attempts to publish objects to catalogs unless explicitly skipped
-
When you publish objects from Data Virtualization to a catalog, Data Virtualization now always attempts to publish any duplicate asset to the target catalog. You can override this behavior by either selecting Skip publish if assets already exist in the catalog or by passing
"allow_duplicates": falsein the REST API payload. If duplicates already exist and you do not select to skip, then what occurs next is determined by the Duplicate asset handling setting on the catalog. Catalogs are configured with Update original assets by default, where existing copies are updated while preserving governance annotations like business terms and data classes. To change the duplicate asset setting, see Duplicate asset handling methods.See also Publishing virtual data to a catalog with Data Virtualization.
- Apply consistent asset names across Data Virtualization assets in catalogs and projects
- When you publish objects from Data Virtualization to
catalogs and projects, the resulting asset name no longer includes a schema prefix, which ensures
consistent naming across Cloud Pak for Data. This change
minimizes the possibility of duplicate assets when you import, enrich, and update assets that are
published from Data Virtualization. In addition, you can
also remove schema prefixes from your Data Virtualization
asset names across all your catalogs, projects, and spaces by running the REMOVESCHEMAFROMASSETS
stored procedure. When you run this procedure, Data Virtualization scans all the existing assets and renames them
without the specified schema prefix.
To rename your assets to remove schema prefixes, see the REMOVESCHEMAFROMASSETS stored procedure.
- Control how connections are mapped during import operations with connection ID (CID) mapping management
- Before you import your Data Virtualization objects, you can now choose to create new CID mappings or update how existing connections are mapped from your source environment to your target environment by running the MANAGE_CID_MAPPING stored procedure. You can run this procedure as part of both the Git based and Data Virtualization API based migration workflows.
- Use OAuth 2.0 authentication when you create a REST API connection
- You can now use the Open Authorization (OAuth) 2.0 authorization protocol for authentication when you create a new REST API connection from the Data Virtualization web client. This option is now available in addition to existing Basic Authentication and Bearer Token options.
- Deny access to virtual tables while masking the derived views with IBM® Knowledge Catalog data protection rules
- You can now use IBM Knowledge Catalog data protection rules to deny a user access to virtual tables while masking the derived views. You can apply deny and masking rules simultaneously and follow the Db2 authorizations plus RCAC model applicable to views.
- Deny access to objects that are not published to a governed catalog by enabling the Restrict access to ungoverned objects setting
- Data Virtualization Administrators can now enforce
governance on all objects by configuring Data Virtualization to deny access to any objects that are not
published to a governed catalog.
To enable the Restrict access to ungoverned objects setting, see Allowing and denying access to data in Data Virtualization.
- Caches now automatically clear when you change your personal credentials setting
- You can now change your credential settings from personal to shared credentials, or vice versa, even after creating virtual tables or caches, without manually removing any caches linked to the connection.
- Use Instana® for better data observability
- You can now send your metrics to Instana to monitor and analyze Data Virtualization. To enable Instana on your Data Virtualization service or on each of your Data Virtualization instances.
- Updates
- The following updates were introduced in this release:
- Migrate your connections as part of the Git migration workflow
- You can now migrate your data source connections that use shared credentials.
- Customize watchdog suspension times by using the
WATCHDOG_SUSPEND_DURATION_MSconfiguration - Administrators can now use the
WATCHDOG_SUSPEND_DURATION_MSconfiguration to control how long the Data Virtualization watchdog pauses its liveliness checks on Derby (GaianDB) during SSL certificate reloads from its default value of 15 seconds when a new certificate is added to GaianDB truststore. Restarting GaianDB is not required.To configure
WATCHDOG_SUSPEND_DURATION_MS, see SETCONFIGPROPERTY properties.
- Update to the audit fields in Zen Audit
- Three fields are now added to Zen Audit for Data Virtualization events:
chkauthidfield which captures the primary authorization ID that is associated with the action being audited.- The
correlation_id fieldis now split into two separate fields:transaction_idandcorrelator. These fields enhance the cross-category correlation and provide better traceability in the system.Note:correlation_idis now deprecated.
- Capture and determine which transaction events rollback
- When auditing of SQL statements is enabled and a Db2 transaction is rolled back, a separate event
data-virtualization.stmt.rollbackis also logged. You can then correlate the transaction events to identify which events were rolled back by using thetransaction_idfield.See the STMT category in Db2 audit events for Data Virtualization.
- Improve the loading performance of Presto by using schema and table filters
- You can now use schema and table filters to improve the loading times for Presto tables and columns that contain the underscore (_) character, the percent (%) character, or both.
- Issues fixed in this release
- The following issues were fixed in this release:
- RCAC data protection rules are not applied to a view if both Deny access and masking IBM Knowledge Catalog rules are applied to the table referenced by the view
- If a table has both the Deny access rule and external RCAC (column and row filters) IBM Knowledge Catalog data protection rules applied to it, then the RCAC rules are not applied to any views that reference that table, leaving the view unmasked.
- Different parameters are displayed for the Apache Spark SQL connector
- If you create a new Apache Spark SQL connection from the platform web client and do not apply any filters, the Apache Spark SQL connector displays different parameters in the platform web client compared to the web client in Data Virtualization.
- Audit runs generate an ignorable error entry
- Every audit run generates an additional (false) error entry in the diagnostic logs, indicating a path access permission issue. You can safely ignore this error.
- Incomplete audit logging when stopping Big SQL or upon Db2 termination
- If you run the
bigsql stopcommand, or in the case of an abrupt unhandled Db2 termination (crash), any statements or events that are in buffer but had not been written to disk yet are not captured in the audit logs. Buffer size can be configured by using the audit_buf_sz configuration parameter.
- Encoding detection override for files data with Japanese characters
- For text files exposed by remote connectors, Cloud Pak for Data automatically detects the encoding scheme of flat data files, such as CSV and TSV files. However, to avoid decoding issues, it is recommended that you set the encoding scheme manually for flat data files.
- When you virtualize data that contains LOB (CLOB/BLOB) or long VARCHAR data types, the preview might show the columns as empty
- After you virtualize the table, in Virtualized data, the data is available for the columns that contain LOB or long VAR CHAR data types.
- Unable to create tables for SSL Cloud Object Storage due to certificate issue
- If you use a custom SSL Cloud Object Storage connection, then you will not be able to perform virtualization operations on that connection without completing a workaround.
- Removing and then recreating an existing Cloud Object Storage or Amazon S3 connection might cause virtualization issues
- If you create, use, delete and then re-create a Cloud Object Storage or Amazon S3 connection, then you might no longer be able to virtualize files from that connection.
- Customer-reported issues fixed in this release
- For a list of customer-reported issues that were fixed in this release, see the Fix List for IBM Cloud Pak for Data on the IBM Support website.