Known issues and limitations for Data Virtualization

The following known issues and limitations apply to Data Virtualization.

For additional solutions to problems that you might encounter with Data Virtualization, see the troubleshooting topic Troubleshooting the Data Virtualization service.

General issues
The following general issues impacts Data Virtualization users.
Audit issues
The following issues and limitations apply to audit issues in Data Virtualization.
Data source and connection issues and limitations
Installation and upgrade issues
The following issue applies to installing and upgrading Data Virtualization. For more information, see Installing Data Virtualization and Upgrading Data Virtualization.
Data virtualization issues
The following issues and limitations apply to virtualization in Data Virtualization.
User and group management issues
The following issues and limitations apply to user and group management in Data Virtualization. For more information about user and group management, see Virtualizing data.
Data governance issues
The following issues and limitations apply to data governance in Data Virtualization.
Caching issues
The following issues and limitations apply to caching in Data Virtualization.
File issues
The following issues and limitations apply to files in Data Virtualization.

General issues

Disabling the platform admin user causes alert monitor to report Dummy evaluation against IKC DPS failed error

Applies to: 5.4.0

If you disable the platform admin user after completing the steps in Disabling the default platform administrator, the Data Virtualization instance IBM Knowledge Catalog integration health check alert monitor starts reporting the Dummy evaluation against IKC DPS failed error.

Workaround: To workaround this issue, complete one of these three options.
  • Ignore the Dummy evaluation against IKC DPS failed error, because it is a false alarm in this case. However, there might be instances where the failure is a real error that should not be ignored.
  • Edit the script by following these steps:
    Note: You will need to re-apply this change if the pod restarts.
    1. Run this command to log into your project namespace: 
      oc project ${DV_NAMESPACE}
    2. Run this command to edit the script: 
      oc rsh c-db2u-dv-db2u-0 bash -c "su - db2inst1 -c \"sed -i 's/ADMIN/__connect_sdk__/g' /opt/dv/current/run_time/wkc/ikc_integration_health_check.sh\""
      After you complete this step, wait for the alert monitor to update (every 10 minutes) to view an accurate status report.
  • Add a regular user named admin to IBM Software Hub, even if the user has no privileges. After you complete this step, wait for the alert monitor to update (every 10 minutes) to view an accurate status report.
Unlocking a Data Virtualization connection incorrectly requires an access token after specifying to use your platform login credentials

Applies to: 5.4.0

When you use a Data Virtualization connection in catalogs or projects, you might be prompted to unlock the connection with your own personal credentials. If you select Use my platform login credentials, the Access token input field is incorrectly marked as required, and you are unable to connect.

Workaround: Open the Data Virtualization connection asset explicitly in the current catalog or project (Platform connections) and then select Use my platform login credentials. You will not be prompted to provide an Access token. Save the changes.

Running the NUMTABSREMOTE view for multiple catalog sources returns -1 as the total number of tables

Applies to: 5.4.0

When you run the NUMTABSREMOTE view to retrieve the total number of tables for multiple catalog sources, the output returns -1 and this is expected. The console displays xxx/? with xxx as the placeholder for the number of tables in the selected catalog. For example: 123/?.

Audit issues

Audit functionality might not be updated after you upgrade to Data Virtualization version 5.4.0

Applies to: 5.4.0

After you upgrade Data Virtualization to IBM Software Hub version 5.4.0, the audit functionality might not be updated.

Symptoms:
  1. The /mnt/blumeta0/support/audit_installation.log log file in the Data Virtualization head pod c-db2u-dv-db2u-0 contains the following message: SQL1024N A database connection does not exist. After upgrading to IBM Software Hub version 5.4.0 from a version prior to 5.4.0, the required table AUDIT.TMP_STMT_TRANSACTIONS is expected to exist. If the table does not exist, then it indicates that the upgrade was not successful or was incomplete. You can check for the existence of the table by running SELECT * AUDIT.TMP_STMT_TRANSACTIONS.
Workaround: As a Data Virtualization Administrator, use the DVSYS.TOGGLEDEFAULTAUDITLOGGINGCOMPLETE procedure to reinstall the Db2 audit facility. See also Enabling and disabling the Db2 audit facility in Data Virtualization.
Note: This workaround resets all the changes made to the default Data Virtualization audit policy named DV_AUDIT_DEFAULT.
  1. Reinstall the audit facility by running this procedure:
    CALL DVSYS.TOGGLEDEFAULTAUDITLOGGINGCOMPLETE('ON')
Incomplete audit logging due to abrupt pod termination

Applies to: 5.4.0

If a pod is abruptly terminated, for example due to out-of-memory error, any statements or events that are still in the in-memory buffer and have not yet been written to disk will not be captured in the audit logs. You can configure the buffer size by using the audit_buf_sz configuration parameter.

CHECKING audit category might generate duplicate entries

Applies to: 5.4.0

If a user runs a statement through a JDBC or ODBC connection on a table for which they do not have access privileges, the CHECKING audit category might generate duplicate entries.
Note: This issue occurs only for statements executed through JDBC/ODBC connections. Statements that are ran using the Db2 CLI work as expected and do not produce duplicate entries.
Audit log entries might not be forwarded to the zen-audit service

Applies to: 5.4.0

Under certain conditions, such as a log‑streaming pod downtime or issues with the audit log PVC, audit log entries might not be forwarded to the IBM Software Hub audit service during the affected period.

Data source and connection issues

For information about limitations and known issues that apply to data sources and connections, see Limitations and known issues for data sources in Data Virtualization.

Data virtualization issues

Incorrect column names might appear when you preview joined virtualized assets
Applies to: 5.4.0

When you join two virtualized assets, and then preview it, the column names displayed might be different from those in the original assets. This naming discrepancy impacts only the preview, and does not effect the final virtualized data objects, which will display the correct column names. There is no workaround.

RELOADTABLES API and the List view might be slow or might fail to fetch data if the schema, table, or column name contains wildcard characters
Applies to: 5.4.0

You might not see tables after you run the RELOADTABLES API or access the List view from the Virtualize page. When a schema, table, or column name contains wildcard characters such as an underscore (_) or percent sign (%), a wildcard search is performed instead of a literal search when fetching data. As a result, the execution phase might be slow or might fail in some cases.

Workaround: Configure schema filtering to limit the scope and speed up RELOADTABLES. You can set the RELOADTABLES_ALLTABS and RELOADTABLES_ALLCOLS filters using the SETCONFIGPROPERTY API on the Run SQL page. For schema, table, or column names that contain wildcards, ensure that you escape the wildcard characters using two backslashes (\). See also SETCONFIGPROPERTY properties.
In this example, hive_data is the catalog name and gosales_1021 is the schema but with an escaped underscore (gosales\\_1021):
call DVSYS.setconfigproperty('RELOADTABLES_ALLTABS_PREST10000','hive_data,gosales\\_1021,null,null','qpendpoint_1',?,?);
call DVSYS.setconfigproperty('RELOADTABLES_ALLCOLS_PREST10000','hive_data,gosales\\_1021,null,null','qpendpoint_1',?,?);
Users with a username that start with an underscore (_) are unable to access or query virtualized tables
Applies to: 5.4.0
Despite being granted the required permissions, users that have a username that start with an underscore (_) are unable to access or query virtualized tables. If that user attempts to access virtualized tables, then this error displays:
Unexpected error code "58009" received from data source "QPLEX". Associated text and tokens are "Network protocol exception: DSS chained with same id at en".. SQLCODE=-1822, SQLSTATE= , DRIVER=4.32.28

Workaround: Create users with a username that do not begin with an underscore (_), then proceed to grant them permissions, and then attempt to access or query the tables again.

The List view might display additional tables
Applies to: 5.4.0
If you added a datasource with personal credentials enabled on the platform connection, then the List view in the Virtualize page might display:
  • All of the tables that the user has access to; however, these tables are not available for previewing or virtualizing.
  • Additional tables that the user might not have access to.
The List view might not display the column name or type of some tables
Applies to: 5.4.0
If you added a datasource with personal credentials enabled on the platform connection, then the column name and type metadata of some tables might not be displayed in the List view. However, you can still virtualize those tables and modify the metadata of the virtualized table by selecting Edit columns from the Review cart and Virtualize tables page.
The List view on the Virtualize page does not consistently display tables or views
Applies to: 5.4.0
The List view does not accurately display the tables or views the user has access to for data sources that have personal credentials enabled. For data sources with personal credentials enabled, this issue occurs because the list view displays tables or views that are relevant to CCDEFINER user.
Workaround: Use the Explore view to view the tables and views that you have access to.
Recreating virtualized tables after using the DROP NICKNAME SQL statement results in object already exists error

Applies to: 5.4.0

When you use the DROP NICKNAME SQL statement to drop tables that have a defined nickname, attempting to recreate and virtualize the tables in the Review cart and virtualize tables page results in an Object already exists error.
Workaround: Use the delete API to remove all the data that is associated with the virtualized tables, then attempt to virtualize the table again using the same nickname. The delete API does not indicate whether it ran successfully, and you might encounter an error. Regardless, you should attempt to virtualize the tables again. To use the delete API, you will need to get an authorization token. For more information, see Delete virtualized data and Get authorization token.
Data that is virtualized with double-byte coded character sets and special characters might appear truncated

Applies to: 5.4.0

When you virtualize a table with a double-byte coded character set and special characters, such as Swedish characters, data might appear to be cut off.

Workaround: When virtualizing a table, use the Edit columns menu to extend the field lengths for those columns.

Virtualizing a table with many columns or long column names might fail

Applies to: 5.4.0

Virtualizing some tables in Data Virtualization might fail with a Virtual table creation failed message. Some tables contain columns with large STRING and CLOB data. Virtualizing the table by reducing the column width or even choosing just one of the columns fails with an error message.

Workaround:
  1. Set the remote schema filter to a schema where the large table does not exist so that it does not get cached.
  2. Create the virtual table with a direct CREATE OR REPLACE NICKNAME statement that includes the SOURCELIST definition option as shown in the following example. The SOURCELIST value describes the remote location of the table to be virtualized in the format <CID>:<RemoteSchema>. The CID is the connection ID for the source and can be found in source details of the Data sources page, or in the result of select * from dvsys.listrdbc query.
    /* IBM_DVSYS */ CREATE OR REPLACE NICKNAME "ADMIN"."VERY_LARGE_TABLE" for QPLEX.gaiandb."VERY_LARGE_TABLE"
    ( 
        VERY_LONG_COLUMN_IDENTIFIER1,
        VERY_LONG_COLUMN_IDENTIFIER2,
        VERY_LONG_COLUMN_IDENTIFIER3,
        ...
        VERY_LONG_COLUMN_IDENTIFIER999,
        VERY_LONG_COLUMN_IDENTIFIER1000
) OPTIONS(SOURCELIST 'DB210000:DRV');
    An example of an SQL command to create a large virtual table.
  3. Query individual columns of the large virtual table.

    An example of an SQL statement to query a large table.

  4. Run aggregations and most other queries.
    Note:

    You cannot select all columns for the large table with a SELECT * statement. You see an error message that is similar to the following message.

    SQL5105N  The statement failed because a Big SQL component encountered an 
error.  Component receiving the error: "DV-FMP".  Component returning the 
error: "Virtualization Connector".  Log entry identifier: "GAI-001-NA".  
Reason: "".  SQLSTATE=58040

Installation and upgrade issues

The nodegroup name of tablespace BIGSQLCATUTILITY changes after upgrading from version 5.2.2 or 5.3.1 to 5.4.x
5.4.0

When you upgrade Data Virtualization on IBM Software Hub from version 5.2.2 or 5.3.1 to 5.4.x, the nodegroup (NGNAME) for tablespace (TBSPACE) BIGSQLCATUTILITY changes from BIGSQLCATGROUP to IBMCATGROUP. Both BIGSQLCATGROUP and IBMCATGROUP node groups are identical, and do not impact the functionality of Data Virtualization.

Pre-upgrade example:
 ============================================================
== B00. SYSCAT.TABLESPACES
============================================================
select varchar(ngname,20)ngname,varchar(tbspace,20)tbspace from syscat.tablespaces order by 1,2

NGNAME               TBSPACE
-------------------- --------------------
BIGSQLCATGROUP       AUDITSPACE
BIGSQLCATGROUP       BIGSQLCATSPACE
BIGSQLCATGROUP       BIGSQLCATUTILITY              <<<< NGNAME is BIGSQLCATGROUP using TBSPACE  BIGSQLCATUTILITY
IBMCATGROUP          DVUSERTEMP
IBMCATGROUP          SYSCATSPACE
IBMCATGROUP          SYSTOOLSPACE
IBMCATGROUP          SYSTOOLSTMPSPACE
IBMCATGROUP          TS4CONSOLE
IBMCATGROUP          TS4MONITOR
IBMDEFAULTGROUP      BIGSQLUTILITYSPACE
IBMTEMPGROUP         TEMPSPACE1

  11 record(s) selected.
Post-upgrade to 5.4.x example:
============================================================
== B00. SYSCAT.TABLESPACES
============================================================
select varchar(ngname,20)ngname,varchar(tbspace,20)tbspace from syscat.tablespaces order by 1,2

NGNAME               TBSPACE
-------------------- --------------------
BIGSQLCATGROUP       AUDITSPACE
BIGSQLCATGROUP       BIGSQLCATSPACE
IBMCATGROUP          BIGSQLCATUTILITY              <<<< NGNAME is IBMCATGROUP using TBSPACE  BIGSQLCATUTILITY
IBMCATGROUP          DVUSERTEMP
IBMCATGROUP          SYSCATSPACE
IBMCATGROUP          SYSTOOLSPACE
IBMCATGROUP          SYSTOOLSTMPSPACE
IBMCATGROUP          TS4CONSOLE
IBMCATGROUP          TS4MONITOR
IBMDEFAULTGROUP      BIGSQLUTILITYSPACE
IBMTEMPGROUP         TEMPSPACE1

  11 record(s) selected.
listnodes does not display remote connects after upgrade from version 5.2.0 to 5.4.0

Applies to: 5.4.0

After you upgrade to IBM Cloud Pak® for Data version 5.4.0, all of the remote connectors associated with the Data Virtualization instance does not show up in listnodes when you run select node_name from dvsys.listnodes. Only the qpagents are displayed. Upgrading remote connectors by using the stored procedure causes the following error, preventing the remote connectors from being updated automatically.

Example error:

[db2inst1@c-db2u-dv-db2u-0 - Db2U ~]$ db2 "call dvsys.updateremoteconnector('apiocp418fips1ucpfyreibmcom:6556',?,?)"

  Value of output parameters
  --------------------------
  Parameter Name  : NUM_UPGRADED
  Parameter Value : 0

  Parameter Name  : DIAGS
  Parameter Value : null,null, failed with The number of columns in the derived column list must match the number of columns in table 'GQ'.;

  Return Status = 0
[db2inst1@c-db2u-dv-db2u-0 - Db2U ~]$
Workaround: Update the remote connectors manually by completing these steps.
  1. On the remote connector machine, run the following command from the sysroot sub-directory inside of your remote connector directory:
    killGaianServers.sh
  2. Create a new remote connector install script from the IBM Cloud Pak for Data console. Use a temporary directory on the Remote Connector machine as your target install directory for this step.
  3. Run the newly created remote connector install script. This script downloads the latest remote connector release and installs it in your chosen temporary directory.
  4. Promptly stop the new remote connector:
    Linux
    On Linux, run the killGaianServer.sh script.
    Windows
    On Windows, run the uninstall_service.bat file.
  5. Copy the contents of sysroot/lib in your new temporary directory to your original remote connector directory sysroot/lib to update the old remote connector jars.
  6. Start your original remote connector:
    Linux
    On Linux, run this command: 
    nohup datavirtualization_start.sh
    Windows
    On Windows, start the DataVirtualizationService service. For example: DataVirtualizationService6414.
Upgrading your Data Virtualization instance from IBM Cloud Pak for Data version 4.8 and 5.1.2 displays Error updating Db2 database error

Applies to: 5.4.0

When you attempt to upgrade your Data Virtualization instance to 5.4.0, the upgrade fails and displays the Error updating Db2 database error in the head pod similar to this example:
2025-06-07T18:53:46.044414567Z MESSAGE: Failed to create module SYSIBMADM.IDAX_MESS_P_RAISE_MESSAGE so forced to recreate all objects in the array
2025-06-07T18:53:46.044414567Z MESSAGE: Error creating module: SYSIBMADM.IDAX_MESS_P_RAISE_MESSAGE
... ...
2025-06-07T18:53:46.156792202Z The database update (db2updv121) exited with a non-zero return-code 72, See /db2u/tmp/db2updv.BIGSQL.20250607184055.log for more details ...
2025-06-07T18:53:46.156792202Z + return 0
2025-06-07T18:53:46.156792202Z + exit 72
2025-06-07T18:53:46.159809670Z + [[ 72 -ne 0 ]]
2025-06-07T18:53:46.159809670Z + log_echo bigsql_update_or_upgrade_db 'Error updating Db2 database'
Additionally, Db2 query indicates that there were no ROUTINE with name IDAX_MESS:
[db2inst1@c-db2u-dv-db2u-0 - Db2U ~]$ db2 "select varchar(routineschema,10)routineschema, varchar(routinename,25)routinename , varchar(specificname,35)specificname from syscat.rodb2 "select varchar(routineschema,10)routineschema, varchar(routinename,25)routinename , varchar(specificname,35)specificname from syscat.routines where routinemodulename = 'IDAX' and (routinename like 'IDAX_MESS%' or specificname like 'IDAX_MESS%' )"

ROUTINESCHEMA ROUTINENAME               SPECIFICNAME
------------- ------------------------- -----------------------------------

  0 record(s) selected.
However, the worker pods are upgraded successfully:
2025-06-07T18:36:04.195966791Z + echo -e '/opt/dv/current/db2u-dv-setup.sh 2025-06-07_18.36.04.191_UTC  INFO db2u-dv-setup,PHASE:end,MSG:Start up DV on c-db2u-dv-db2u-1.c-db2u-dv-db2u-internal.thanos.svc.cluster.local'
2025-06-07T18:36:04.200718069Z + ec=0
Workaround: Scale up or scale down the Data Virtualization head and worker pods to restart the pods and resume the upgrade. Replace instances of <DV_INSTANCE_NAMESPACE> with your Data Virtualization instance namespace.
  1. Obtain the Data Virtualization head and worker sts current replicas number:
    DV_STS_REPLICAS=$(oc -n <DV_INSTANCE_NAMESPACE> get sts c-db2u-dv-db2u -ojsonpath={.spec.replicas})
  2. Scale down the Data Virtualization head and worker pods: 
    oc -n <DV_INSTANCE_NAMESPACE> scale sts c-db2u-dv-db2u --replicas=0
    Wait for all of the c-db2u-dv-db2u-xxx pods to delete.
  3. Scale up the Data Virtualization head and worker pods to the replicas that you obtained from the first step:
    oc -n <DV_INSTANCE_NAMESPACE> scale sts c-db2u-dv-db2u --replicas=$DV_STS_REPLICAS
    The Data Virtualization head and worker pods restart and the Data Virtualization upgrade resumes automatically.
Data Virtualization passthrough route c-db2u-dv-db2u is missing after upgrade

Applies to: 5.4.0

When you upgrade your Data Virtualization instance, the passthrough route is missing and Data Virtualization displays the error FAIL: route c-db2u-dv-db2u does not exist in namespace zen.
The error might resemble this example:
=======================================================================
dv-validate.sh : 12. Check able to connect to DV port
=======================================================================

Tue Jun 3 08:02:02 UTC 2025 12. Check able to connect to DV SSL port

$ oc get --namespace zen -o jsonpath='{.spec.host}' route c-db2u-dv-db2u
Error from server (NotFound): routes.route.openshift.io "c-db2u-dv-db2u" not found
dv-validate.sh : FAIL: route c-db2u-dv-db2u does not exist in namespace zen
$ date
Tue Jun  3 08:02:03 UTC 2025
Symptoms: Running this command does not return a route.
  • Replace <DV_INSTANCE_NAMESPACE> with your Data Virtualization instance namespace. 
oc -n <DV_INSTANCE_NAMESPACE> get route c-db2u-dv-db2u
Workaround: To workaround this issue, you can either use NodePort instead of the passthrough route, or you can manually create the route after the Data Virtualization instance upgrade. For more information on NodePort, see Configuring network requirements.

To manually create the route, complete these steps:

  1. Set the DV_NAMESPACE variable with the OpenShift® namespace that contains the Data Virtualization instance.
  2. Set the EXTERNAL_HOST variable with the external host name accessible to the end users.
  3. Run this command to obtain the Data Virtualization formation ID.
    DV_FORMATION_UID=$(oc -n $DV_NAMESPACE get formation db2u-dv -o jsonpath="{.metadata.uid}")
  4. Run this command to create the route in the same namespace as you set in the DV_NAMESPACE variable. The name of the route is c-db2u-dv-db2u.
    cat <<EOF | oc apply -f -
apiVersion: route.openshift.io/v1
kind: Route
metadata:
  annotations:
    openshift.io/host.generated: "true"
  labels:
    app.kubernetes.io/component: cr
    app.kubernetes.io/name: db2u-dv
    icpdsupport/addOnId: dv
    icpdsupport/module: instance
  name: c-db2u-dv-db2u
  namespace: $DV_NAMESPACE
  ownerReferences:
  - apiVersion: db2u.databases.ibm.com/v1
    kind: Formation
    name: db2u-dv
    uid: $DV_FORMATION_UID
spec:
  host: c-db2u-dv-db2u-$DV_NAMESPACE.$EXTERNAL_HOST
  port:
    targetPort: 50001
  tls:
    termination: passthrough
  to:
    kind: Service
    name: c-db2u-dv-dv-head-svc
    weight: 100
  wildcardPolicy: None
EOF
  5. Verify the new route by running this command:
    oc -n $DV_NAMESPACE get route c-db2u-dv-db2u
    This command should return one route.
Upgrading remote connectors from Cloud Pak for Data version 4.8.8 fails with SQL4302N error

Applies to: 5.4.0

When you upgrade your Data Virtualization remote connectors from Cloud Pak for Data version 4.8.8 to 5.4.0, the dvsys.updateremoteconnector() procedure fails and displays the SQL4302N error.

This issue is due to jars in the /mnt/blumeta0/home/db2inst1/sqllib/function directory not being upgraded correctly.

The error might resemble this example:
select node_name,agent_class from dvsys.listnodes where agent_class = 'R' 

NODE_NAME                                                                        AGENT_CLASS         
-------------------------------------------------------------------------------- --------------------
apiocp455fips6rcpfyreibmcom:6556                                                 R                   

  1 record(s) selected.


call dvsys.updateremoteconnector('',?,?) 
SQL4302N  Procedure or user-defined function "DVSYS.UPDATEREMOTECONN", 
specific name "DV_UPDATEREMOTEGAIANNO" aborted with an exception 
"com/ibm/gaiandb/GaianDBC".  SQLSTATE=38501

select node_name,agent_class from dvsys.listnodes where agent_class = 'R' 

NODE_NAME                                                                        AGENT_CLASS         
-------------------------------------------------------------------------------- --------------------
apiocp455fips6rcpfyreibmcom:6556                                                 R                   

  1 record(s) selected.
Workaround:
  1. Log into the Data Virtualization head pod and switch to the db2inst1 user:
    oc ${DV_INSTANCE_NAMESPACE} rsh c-db2u-dv-db2u-0 bash
su - db2inst1
  2. In the sqllib/function directory, complete the following:
    1. Delete the dv_caching_predict.py file.
    2. Delete the model_rr_d3_d3_1g.pkl file.
    3. Delete all of the jar files except for the BIDDL.jar file.
  3. Restart Db2 by running these commands:
    1. db2stop force
    2. db2start
  4. On the remote connector host, install Java 21 from IBM Semeru Runtimes Downloads.
  5. Open the datavirtualization.env file, and replace the existing JAVA_HOME path with the new Java 21 path.
  6. In the Data Virtualization head pod as the db2inst1 user, run this command:
    dvsys.updateremoteconnector()
  7. On the remote connector host, navigate to the sysroot/data directory and complete the following:
    1. In the derby.properties file, change the derby.authentication.provider property to com.ibm.gaiandb.queryplex.DVAuthenticator.
    2. In the gaiandb_config.properties file, change the EXTENDER_CLASS_NAME property to com.ibm.gaiandb.queryplex.ConfluenceExtender.
  8. Start the remote connector:
    • Linux
      On Linux, run this command: 
      nohup datavirtualization_start.sh
    • Windows
      On Windows, start the DataVirtualizationService service. For example: DataVirtualizationService6414.
  9. In the Data Virtualization head pod as the db2inst1 user, verify the remote connector by running these commands:
    Note: In addition, queries that you created using the remote connector before the upgrade should now work again. 
    1. db2 connect to bigsql
    2. db2  "select node_name from dvsys.listnodes"
      The result displays the endpoints and the remote connector nodes, similar to this example:
      NODE_NAME                                                                       
--------------------------------------------------------------------------------
AdminNode                                                                       
apiocp455b1ccpfyreibmcom:6556                                                   
qpendpoint_1:6415                                                               
qpendpoint_2:6416                                                               
qpendpoint_3:6417                                                               
qpendpoint_4:6418                                                               
qpendpoint_5:6419                                                               

  7 record(s) selected.
Upgrading remote connectors from Cloud Pak for Data version 5.0 fails with Failed to upgrade GaianDB jars null error

Applies to: 5.4.0

When you upgrade your Data Virtualization remote connectors from Cloud Pak for Data version 5.0, the upgrade fails with the Failed to upgrade GaianDB jars null error.

The error resembles the following example:
select node_name,agent_class from dvsys.listnodes where agent_class = 'R' 

NODE_NAME                                                                        AGENT_CLASS         
-------------------------------------------------------------------------------- --------------------
apiocp417fips6ecpfyreibmcom:6556                                                 R                   

  1 record(s) selected.


call dvsys.updateremoteconnector('',?,?) 

  Value of output parameters
  --------------------------
  Parameter Name  : NUM_UPGRADED
  Parameter Value : 0

  Parameter Name  : DIAGS
  Parameter Value : apiocp417fips6ecpfyreibmcom:6556, failed with The exception 'java.lang.Exception: Failed to upgrade GaianDB jars null' was thrown while evaluating an expression.;

  Return Status = 0

select node_name,agent_class from dvsys.listnodes where agent_class = 'R' 

NODE_NAME                                                                        AGENT_CLASS         
-------------------------------------------------------------------------------- --------------------

  0 record(s) selected.
Workaround: Complete these steps to workaround the issue.
  1. On the remote connector machine, run the following command from the sysroot sub-directory inside of your remote connector directory.
    killGaianServers.sh
  2. Create a new remote connector install script from the Cloud Pak for Data console. Use a temporary directory on the Remote Connector machine as your target install directory for this step.
  3. Run the newly created remote connector install script. This script downloads the latest remote connector release and installs it in your chosen temporary directory.
  4. Promptly stop the new remote connector.
    • Linux

      On Linux, run the killGaianServer.sh script.

    • Windows
      On Windows, run the uninstall_service.bat file.
  5. Copy the contents of sysroot/lib in your new temporary directory to your original remote connector directory sysroot/lib to update the old remote connector jars.
  6. Start your original remote connector.
    • Linux
      On Linux, run this command: 
      nohup datavirtualization_start.sh
    • Windows
      On Windows, start the DataVirtualizationService service. For example: DataVirtualizationService6414.
Upgrading existing Cloud data source connections to Data Virtualization on Cloud Pak for Data version 5.4.0 causes certificate errors

Applies to: 5.4.0

When you upgrade an existing Cloud data source connection from a release prior to 5.1.0 to Data Virtualization on Cloud Pak for Data version 5.4.0, you might encounter this error stating that the certificate cannot be found:
... Reason: /opt/ibm/db2/V11.5.0.0/java/jdk64/jre/lib/security/cacerts (No such file or directory)
In releases prior to version 5.1.0, certificates that were required to connect to an external data source were stored in the default Java trust store. In version 5.1.0, additional security requirements (including FIPS support) require certificates to be stored in trust stores specific to each data source connection. As a result, when you upgrade to version 5.4.0, existing connections might continue to point to the old Java trust store instead of the required trust store for each data source.

Workaround: To workaround this issue, complete these steps.
  1. Run this stored procedure to recreate all the connections using the newly updated connection information.
    call DVSYS.RESTORECONFIG('',?,?)
    Ensure that all the connections (CID) are listed as AVAILABLE on the web client. If this does not fix the problem, or there remains UNAVAILABLE connections, then proceed to the next step.
  2. Conditional step: If the previous step did not fix the problem, then complete this conditional step in the RunSQL console.
    1. Run this query to retrieve the CID of all UNAVAILABLE connections.
      select CID from DVSYS.listrdbc where cstatus='UNAVAILABLE';
      Take a note of the CID for the next step.
    2. Run this query to retrieve the CCID, USR and PWD. Replace <CID> with the CID from the previous step.
      Note: You must repeat these steps for each UNAVAILABLE connection.
      select * from  dvsys.connection_details where cid='<CID>'"
      For example: 
      select * from  dvsys.connection_details where cid='DB210000'"
      If the connection uses vaulted credentials, the output displays the user and password masked using the string {SECRET_REF}.
    3. Run the following setRdbcX stored procedure. Ensure you replace <SRC_TYPE>, <HOST_NAME>, <DATABASE_PORT>,<DATABASE_NAME>, <USR>, <PWD>, <USE_SSL> and <CCID> values from the connection_details table, and the associated <NODE_NAME> and <CID> values from the first step.

      You must also add additional options including EDITCID and CCID in <ADDITIONAL_OPTIONS>. See the following example.

      Note: If the connection does not use vaulted credentials, then you must explicitly replace <USR> and <PWD> with the correct user and password. This requires a user with the Data Virtualization Admin role. 
      call DVSYS.setRdbcX('<SRC_TYPE>','<HOST_NAME>','<DATABASE_PORT>','<DATABASE_NAME>','','<USR>','<PWD>','<USE_SSL>','<VALIDATE_CERT>','','','<NODE_NAME>','<ADDITIONAL_OPTIONS>',?,?,?);
      For example:
      call DVSYS.setRdbcX('DB2','924186d3-ec79-4daf-aa77-3297081a929b.zu9tc4nd0urletsa6ufg.ds.app.cloud','31101','bludb','','{SECRET_REF}','{SECRET_REF}', '1', '0', '', '', 'qpendpoint_3:6417', 'EDITCID=DB210000,CCID=85b3d676-f627-4f57-a8fd-aad00782c2de',?,?,?)
      The connection is recreated successfully.
Upgrading remote connectors from IBM Cloud Pak for Data versions prior to 5.1.0 causes an Unable to initialise GaianNode data (aborting) error

Applies to: 5.4.0

When you upgrade remote connectors to IBM Cloud Pak for Data version 5.4.0 from versions prior to 5.1.0, using the stored procedure DVSYS.UPDATEREMOTECONNECTOR, the remote connectors do not start and are not displayed in the output select node_name from dvsys.listnodes. An error message similar to the following example can be found in <Remote_Connector_Install_Path>/sysroot/data/gaiandb-stdout.log.

Example error message:

Unable to initialise GaianNode data (aborting): Another GaianNode or Derby instance may be attached to db folder 'gaiandb'? Otherwise, authentication may have failed or the database may be corrupted (delete it & restart to recycle it). Also check optional messages below.
java.sql.SQLException: java.sql.SQLException: Failed to start database 'gaiandb6556' with class loader jdk.internal.loader.ClassLoaders$AppClassLoader@f1fd8dc9, see the next exception for details. SQL: CREATE FUNCTION maintainConnection2 (nodeid VARCHAR(100), usr VARCHAR(10), pwd VARCHAR(10), extraInfo VARCHAR(32672)) RETURNS VARCHAR(32672) PARAMETER STYLE JAVA LANGUAGE JAVA NO SQL EXTERNAL NAME 'com.ibm.gaiandb.GaianDBConfigProcedures.maintainConnection2'
Workaround:
  1. Change the following properties in the files gaiandb_config.properties and derby.properties. Both of the files are located in the directory <Remote_Connector_Install_Path>/sysroot/data.
    • In the gaiandb_config.properties file, replace EXTENDER_CLASS_NAME=com.ibm.gaiandb.confluence.ConfluenceExtender with this property:
      EXTENDER_CLASS_NAME=com.ibm.gaiandb.queryplex.ConfluenceExtender
    • In the derby.properties file, replace derby.authentication.provider=com.ibm.gaiandb.confluence.DVAuthenticator with this property:
      derby.authentication.provider=com.ibm.gaiandb.queryplex.DVAuthenticator
  2. Restart the Remote connector by completing one of the following options.
    • Linux
      For Linux systems, run this command to start the Remote connector from the directory <Remote_Connector_Install_Path>/sysroot: 
      nohup ./datavirtualization_start.sh &
    • Windows
      For Windows systems, open the Microsoft Windows Services console (or services.msc), and then restart the Remote connector. Your Remote connector is named DataVirtualizationService<port> .

      For example, the Remote connector is named DataVirtualizationService6414, and you would restart this service in the Microsoft Windows Services console.

Db2REST pod in CrashLoopBackOff state

Applies to: 5.4.0

You can't connect to the Db2REST server. After enabling REST, the REST pod enters CrashLoopBackOff state and repeatedly restarts and never reaches completion.
You see the following log:
chmod: cannot access '/db2u/tmp/os_envar_configmap': No such file or directory
/db2u/scripts/include/common_functions.sh: line 23: /db2u/tmp/os_envar_configmap: No such file or directory
ls: cannot access '/opt/ibm/dbrest/auth': No such file or directory
......+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*............+..+.+.....+.......+...........+....+..+....+..............+...+...+.......+...+............+...+....................+.+...+..+.........+....+.......................+............+...+....+........+.+.....+.+.....+....+...............+.....+.+.....+......+....+..................+......+..+....+...+..+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
...........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..........+.........+.....+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----
Starting up IBM Db2 REST container...
Starting the Db2 REST server...
Starting server with https
/opt/ibm/dbrest/scripts/../bin/db2rest-server: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory
SERVER STATUS: INACTIVE
#############################################################
###  IBM Db2 REST container was started successfully   ###
#############################################################
* If you used docker/oc/kubectl logs to monitor progress, detach from the
console by pressing Ctrl+C.
* To get a command line prompt, issue the following command from the host:
    [ docker | oc | kubectl -n <NS> ] exec -it <rest container name> bash
#############################################################
Workaround: Complete the following steps to resolve this issue:
  1. Find your affected db2ucluster resource by running the following command:
    oc get db2ucluster
    Refer to the following example output:
    NAME      STATE      AGE
db2u-dv   Ready      34m
  2. Disable REST for your db2ucluster.
    1. Run the following command to edit your db2ucluster:
      oc edit db2ucluster deployment_ID
    2. Set rest to false in your CR. Refer to the following example:
      spec:
  ...
  addOns:
    rest:
      enabled: false
  3. Save your db2u-release Config Map:
    oc get cm db2u-release -n ${PROJECT_CPD_INST_OPERATORS} -o yaml > db2u-release.yaml
  4. Extract, then replace your REST images.
    1. Extract your Db2 11.5.9.0 REST image:
      REST_IMAGE_1159=$(oc get cm db2u-release -n ${PROJECT_CPD_INST_OPERATORS} -o=jsonpath='{.data.json}' | jq '.databases.db2u["11.5.9.0"].images.rest')
    2. Extract your Db2 12.1.0.0 REST image:
      REST_IMAGE_CN1=$(oc get cm db2u-release -n ${PROJECT_CPD_INST_OPERATORS} -o=jsonpath='{.data.json}' | jq '.databases.db2u["12.1.0.0"].images.rest')
    3. Replace your Db2 12.1.0.0 REST image with your 11.5.9.0 REST image:
      sed -i.bak "s|\"rest\": $REST_IMAGE_CN1|\"rest\": $REST_IMAGE_1159|g" db2u-release.yaml
  5. Apply your changes to your ConfigMap:
    oc apply -f db2u-release.yaml -n ${PROJECT_CPD_INST_OPERATORS}
  6. Delete your ConfigMap YAML file:
    rm db2u-release.yaml
  7. Enable REST for your db2ucluster.
    1. Run the following command to edit your db2ucluster:
      oc edit db2ucluster deployment_ID
    2. Set rest to true in your CR. Refer to the following example:
      spec:
  ...
  addOns:
    rest:
      enabled: true

User and group management issues

Cannot manage (Edit, Revoke, or Add) user groups through the Data Virtualization web client

Applies to: 5.4.0

In User management, if you attempt to Edit or Revoke the role of user groups, or you try to Add new user groups, the action fails with these errors:
  • Edit: Unable to update user group role
  • Revoke: Delete stream error
  • Add: method POST is not allowed, but [GET] are
Workaround: To manage user groups, navigate to the Cloud Pak for Data home page > Instances > Data Virtualization > Manage access.
Privileges and authorities that are granted to user groups are not considered when you create views

Applies to: 5.4.0

Db2 privileges (Db2 GRANTS) and authorities that are granted to user groups are not considered when you create views. This limitation is a result of a Db2 limitation on groups.

Workaround: To workaround this issue, instead of granting access to specific groups from the Manage access page, you should grant access to the public by selecting All data virtualization users. Then create a new data protection rule to define Allow or Deny rules based on groups.

Data governance issues

You cannot apply business terms when you virtualize files in data sources on the Files tab

Applies to: 5.4.0

When you virtualize files in Data > Data virtualization > Virtualize, the Business terms column is not available for data sources on the Files tab. These data sources do not support business terms.

You cannot see business term assignments or grouped tables in the Explore view

Applies to: 5.4.0

On the Virtualize page, you cannot see business term assignments and you cannot see grouped tables.

Workaround: Switch to the List view to see business term assignments and grouped tables.

Cannot see list of available tables in the strict virtualization mode

Applies to: 5.4.0

In the strict virtualization mode (where you can see tables only if they have at least one column with business term assignment), when you navigate to the Virtualize page, the console appears to be loading the table list for a while without showing any tables. The loading can be much slower compared to the default virtualization mode while the console evaluates the list of eligible tables that can be virtualized, depending on term assignments to data source table and column names.

A virtualized object cannot be used in Cognos Dashboards without credentials and an appropriate role

Applies to: 5.4.0

Using a virtualized object in Data Virtualization displays an error message if you do not enter credentials for the Data Virtualization connection or you do not have the correct role.

If you did not enter a username and password when you created the connection to the Data Virtualization data source, you see the following error: Missing personal credentials. If you are not assigned the Admin or Steward role for this object, you see the following error: Unable to access this asset.

To work around this issue, see A virtualized object cannot be used in Cognos Dashboards without credentials and an appropriate role in Data Virtualization.

Caching issues

Online backup of Data Virtualization can affect cache refreshes

Applies to: 5.4.0

If you schedule an online backup in Data Virtualization and if at the same time a cache refresh operation runs, the cache write operation can fail.

Online backup in Data Virtualization works by quiescing database write and update operations. Cache refresh is a write operation that updates Data Virtualization caches with fresh data from remote data sources. Both online backup and cache refresh operations can be scheduled to run periodically. These schedules can conflict and every time the cache tries to refresh, an online backup might be in progress or it might start while the cache refresh is in progress. In these scenarios, cache refresh operations can fail with an error similar to the following message. The cache is active but it contains stale data.

The statement failed because a Big SQL component encountered an error. 
Component receiving the error: "BigSQL IO". Component returning the error: "UNKNOWN". 
Log entry identifier: "[BSL-1-b1bf428f]". Reason: "Cannot add block to /warehouse".. 
SQLCODE=-5105, SQLSTATE=58040, DRIVER=4.27.25

Cache auto-refresh might also fail and you can either manually refresh or wait for the next auto-refresh. However, if the online backup is scheduled such that it conflicts with any cache's refresh schedule, then every auto-refresh of those caches will fail.

Workaround: Reschedule the cache's refresh or the online backup schedule to avoid conflicts. You can also refresh the cache manually at any time.

File issues

You might encounter errors when you virtualize large Excel files

Applies to: 5.4.0

You might experience an error when you preview or virtualize large files with Excel format (XLS):
ERROR 400 'Your InputStream was neither an OLE2 stream nor an OOXML stream' was thrown while evaluating an expression.
Workaround: You can follow these steps:
  1. Load the file data into a table on a supported data source.
  2. Virtualize the table.
  3. Optionally, you can change the format of the file; for example, to CSV format before you virtualize it again.