Updating Data Virtualization from Version 4.0.1 or later to 4.0.8

To obtain fixes, security updates, and new functionality, you must upgrade to the latest refresh of Data Virtualization. If Data Virtualization is already installed on top of Cloud Pak for Data Version 4.0, a project administrator can upgrade Data Virtualization to the latest refresh.

For a complete list of Data Virtualization refreshes available on Cloud Pak for Data Version 4.0, see Operator and operand versions.

Permissions you need for this task: You must be an administrator of the OpenShift® project (Kubernetes namespace) where Data Virtualization is installed.

Information you need to confirm before you start this task

Before you update Data Virtualization, confirm the following information:

The name of the project where Data Virtualization is installed.
Data Virtualization is installed in the same project as Cloud Pak for Data.
The storage class or classes that you are using for your existing Data Virtualization installation. The storage must be the same or equivalent to the storage classes listed in Information you need to confirm before you start this task.

Information you need to complete this task

Data Virtualization requires the Cloud Pak for Data common core services. If the common core services are not installed in the project or are not at the correct version, the common core services will be automatically installed when you upgrade Data Virtualization, which will increase the amount of time the upgrade takes to complete.
You cannot change the storage that is associated with this installation. If you try to change the storage, the upgrade will fail.

Before you begin

Ensure that the cluster meets the minimum requirements for Data Virtualization. For details, see System requirements.

Additionally, ensure that a cluster administrator completed the required Upgrade preparation tasks for your environment. Specifically, verify that a cluster administrator completed the following tasks:

Your data is backed up. For more information, see Backing up and restoring an entire deployment with the Cloud Pak for Data OADP backup and restore service.
The latest Cloud Pak for Data refresh is installed. For details, see Upgrading Cloud Pak for Data.
For environments that use a private container registry, such as air-gapped environments, the Data Virtualization software images are mirrored to the private container registry. For details, see Mirroring images to your container registry.
The catalog source for Data Virtualization is updated. For details, see Updating catalog source.
The operator for Data Virtualization is upgraded. For details, see Upgrading operator subscriptions.

If these tasks are not complete, the Data Virtualization upgrade will fail.

Prerequisite services

Before you upgrade Data Virtualization, ensure that the following services are upgraded and running:

Db2U: Update the Db2U catalog source. For more information, see Updating catalog source.
Db2® Data Management Console: Make sure that a Db2 Data Management Console instance has been upgraded and provisioned. For more information, see Upgrading Db2 Data Management Console to a newer refresh on Version 4.0.
Cloud Pak for Data common core services: Data Virtualization installs Common core services on your Cloud Pak for Data cluster if you do not have it installed. Data Virtualization upgrades Common core services if you have it installed.

Procedure

This procedure is a multi-step process. You must upgrade the Data Virtualization service and then upgrade the Data Virtualization instance. Complete the following tasks to upgrade Data Virtualization:

Update the service.
Verify the update.
Update the service instance.
Update remote connectors.
Find out what to do next.

Updating the Data Virtualization service

To upgrade the Data Virtualization service:

Log in to Red Hat® OpenShift Container Platform as a user with sufficient permissions to complete the task:
```
oc login OpenShift_URL:port
```

Edit the DvService custom resource:

oc edit DvService dv-service \
--namespace project-name

Update the version entry to 1.7.8
Save your changes and exit. For example, if you are using vi, hit esc and enter :wq.

When you update the custom resource, the Data Virtualization operator upgrades the Data Virtualization service.

Verifying the update

The Data Virtualization operator continually monitors the contents of the custom resource and updates the microservices that comprise Data Virtualization as needed. The DvService microservice is defined by the dv-service custom resource. The Data Virtualization service is upgraded when the DvService status is True.

To check the status of the upgrade:

Change to the project where Data Virtualization is installed:
```
oc project project-name
```
Get the status of the Data Virtualization custom resource (dv-service):
```
oc get DvService dv-service -o jsonpath='{.status.conditions[?(@.type == "Successful")].status} {"\n"}'
```
Data Virtualization is ready when the command returns True.
Run following command to verify that the level now shows 1.7.8-18.
```
oc -n project-name get pod $(oc get -n project-name pods | grep dv-addon | cut -d' ' -f 1) -o jsonpath={.metadata.annotations.level}
```
Where project-name is the OpenShift project, also known as namespace, where the Data Virtualization service dv-addon and dv-service-provider pods run.

Updating the service instance

After you upgrade Data Virtualization, the service instance that is associated with the installation must also be upgraded. This task must be completed by a Cloud Pak for Data administrator or a service instance administrator.

Run the following commands to delete the existing DV_RIVER_UPGRADED marker file.

oc -n project-name rsh c-db2u-dv-db2u-0 bash

```
su - db2inst1
```

$(db2uctl marker list | grep -i DV_RIVER_UPGRADED 2>&1 > /dev/null);ec=$?; [[ $ec -eq 0 ]] && db2uctl marker delete DV_RIVER_UPGRADED

Delete the /mnt/blumeta0/db2/.upgradedb file from the Data Virtualization head pod if you are upgrading to 1.7.8 from an initial Cloud Pak for Data 3.5.0 cluster.
1. ```
oc -n project-name rsh c-db2u-dv-db2u-0 bash
```
2. ```
su - db2inst1
```
3. ```
sudo rm -rf /mnt/blumeta0/db2/.upgradedb
```
Upgrade the instance by updating its version field:
```
oc patch bigsql db2u-dv --patch '{"spec": {"version": "1.7.8"}}' --type=merge
```
This step triggers the instance update, which will take some time to complete. First, the update runs a backup. If this backup fails, the version in .spec.version is reset to the previous version. This process can be monitored in the logs of the Db2U operator manager pod. The value in .status.version is not updated to match .spec.version until the update process has completed successfully.
Run the following command to verify that the version now reads 1.7.8.
```
oc get bigsql db2u-dv -o jsonpath='{.status.version}{"\n"}'
```
You must wait until the instance update completes before you proceed to the next step.

Update Data Virtualization instance configmap:

oc -n project-name get cm dv-data-virtualization-cm -o yaml > dv-data-virtualization-cm.yaml

sed -i -e "s| \"addon_version\":.*| \"addon_version\": \"1.7.8\",|g" dv-data-virtualization-cm.yaml

sed -i -e "s| icpdata_addon_version:.*| icpdata_addon_version: 1.7.8|g" dv-data-virtualization-cm.yaml

oc -n project-name apply -f dv-data-virtualization-cm.yaml

Where project-name is the OpenShift project where the Data Virtualization pods were created.

Update the Data Virtualization caching pods.
1. Edit the c-db2u-dv-dvcaching pod.
```
oc edit deployment c-db2u-dv-dvcaching
```
2. In the initContainers/env section, search for the following entries, remove the keys and corresponding values, and then save and exit.
  - - name: DB2U_API_KEY_FILE
  - - name: DB2U_API_CERT_FILE
3. Wait for the c-db2u-dv-dvcaching pod to restart. The pod changes from 0/1 Init to 0/1 Running and eventually to 1/1 Running.

Upgrading remote connectors

You can upgrade remote connectors by using the UPDATEREMOTECONNECTOR stored procedure. You can run this procedure by using the SQL editor or the Db2 console on the cluster.

To update all remote connectors, run the following stored procedure.
```
call dvsys.updateremoteconnector('',?,?)
```
If you need to upgrade a set of remote connectors, pass in a comma-separated list.
```
call dvsys.updateremoteconnector('<REMOTE_CONNECTOR_NODES>',?,?)
```
You can obtain the <REMOTE_CONNECTOR_NODES> by running the following command.
```
select node_name from dvsys.listnodes where AGENT_CLASS='R'
```
If you notice that remote connectors do not appear in the user interface after the upgrade, run the following stored procedure on the head pod.
```
CALL DVSYS.DEFINEGATEWAYS('<hostname>:<port>')
```
Where <hostname> is the hostname of the remote connector and <port> is the port number used by the remote connector to connect to Data Virtualization. After you run this stored procedure, the remote connector appears in the user interface and when you run dvsys.listnodes.

See also Defining gateway configuration to access isolated remote connectors.
To troubleshoot issues, see Updating remote connectors might fail with a Java™ exception after you upgrade Data Virtualization.

What to do next

Run the following steps from the Data Virtualization head pod to manually remove unnecessary files.

Log in to the Data Virtualization head pod.
```
oc rsh c-db2u-dv-db2u-0 bash
```
Switch to the db2inst1 user.
```
su - db2inst1
```

Remove unnecessary JAR files.

rm -rf 
/mnt/blumeta0/home/db2inst1/sqllib/datavirtualization/dvm_driver/log4j-core-2.8.2.jar
/mnt/blumeta0/home/db2inst1/sqllib/datavirtualization/dvm_driver/log4j-api-2.8.2.jar
/mnt/bludata0/dv/versioned/pre_migration/sqllib/datavirtualization/dvm_driver/log4j-api-2.8.2.jar
/mnt/bludata0/dv/versioned/pre_migration/sqllib/datavirtualization/dvm_driver/log4j-core-2.8.2.jar

Run the following script to remove the log4j-core-2.8.2.jar file.

${BIGSQL_CLI_DIR}/BIGSQL/package/scripts/bigsqlPexec.sh -w -c "rm -rf /mnt/blumeta0/home/db2inst1/sqllib/datavirtualization/dvm_driver/log4j-core-2.8.2.jar"

Run the following script to remove the log4j-api-2.8.2.jar file.

${BIGSQL_CLI_DIR}/BIGSQL/package/scripts/bigsqlPexec.sh -w -c "rm -rf /mnt/blumeta0/home/db2inst1/sqllib/datavirtualization/dvm_driver/log4j-api-2.8.2.jar"

Remove unnecessary .zip and .tar files.

rm -rf 
/mnt/PV/versioned/uc_dsserver_shared/config/DATAVIRTUALIZATION_ENDPOINT_V1.7*.tar.gz
/mnt/PV/versioned/uc_dsserver_shared/config/DATAVIRTUALIZATION_ENDPOINT_V1.7*.zip

Copy the .tar file for this version.

cp /opt/ibm/qp_artifacts/archives/DATAVIRTUALIZATION_ENDPOINT_V1.7.8_*.tar.gz /mnt/PV/versioned/uc_dsserver_shared/config

Copy the .zip file for this version.

cp /opt/ibm/qp_artifacts/archives/DATAVIRTUALIZATION_ENDPOINT_V1.7.8_*.zip /mnt/PV/versioned/uc_dsserver_shared/config

Complete the following steps to restart head and worker pods.
1. Wait for the Data Virtualization hurricane pod to start successfully.
2. Run the following commands to restart the Data Virtualization head and worker pods:
```
current_replicas=$(oc -n zen get sts c-db2u-dv-db2u -o jsonpath="{.spec.replicas}")
```
```
oc -n zen scale sts c-db2u-dv-db2u --replicas=0; sleep 3m; oc -n zen scale sts c-db2u-dv-db2u --replicas=$current_replicas
```
3. If you see the following error message, restart the Data Virtualization hurricane pod and then repeat step 2.b.
```
ERR api/pkg/cli/sideload/load.go:73 >  error="file is the wrong size: 154274816, expected: 154143232\n"
```
Optionally, set up automatic pruning of the archive log.

Data Virtualization is ready to use. For more information, see Virtualizing data with Data Virtualization.