IBM Support

Data Virtualization service on Cloud Pak for Data appears to be unstable on a Red Hat OpenShift on Kubernetes cluster

Troubleshooting


Problem

In IBM Cloud Pak for Data version 4.0.4 and earlier on a Red Hat OpenShift on Kubernetes Services (ROKS) cluster, you might notice that a Data Virtualization service instance is unstable.

Symptom

You might experience SQL operations intermittently failing, and you might also see SQLCODE=-1229 errors:
  
Error message
The current transaction has been rolled back because of a system error. SQLCODE=-1229, SQLSTATE=40504, DRIVER=4.27.25

Diagnosing The Problem

To confirm the problem, run the following commands:

  1. Log in to Red Hat OpenShift: 
    oc login <OpenShift_URL>:<port>
  2. Change to the project where the Cloud Pak for Data control plane is installed: 
    oc project <Project>
  3. Check whether the ping command is failing: 
    oc exec -ti c-db2u-dv-db2u-0 -- bash -c 'ping -c 1 $(cat /etc/hosts | grep db2u-1 | cut -d" " -f2)'
  4. If the ping command returns the following error, it confirms that the problem is due to a Red Hat OpenShift security constraint: 
    ping: socket: Operation not permitted

Resolving The Problem


To allow the ping operation to work, add the NET_RAW capability to the Data Virtualization instance by running the following commands:

  1. oc get $(oc get scc -o name | grep db2u-dv) -o yaml > c-db2u-dv-scc-bak.yaml

oc get $(oc get sts -l app=db2u-dv,role=db -o name) -o yaml > c-db2u-dv-db2u-bak.yaml

oc patch $(oc get scc -o name | grep db2u-dv) --type=json -p='[{"op": "add", "path": "/allowedCapabilities/0", "value":"NET_RAW" }]'

oc patch $(oc get sts -l app=db2u-dv,role=db -o name) --type=json -p='[{"op": "add", "path": "/spec/template/spec/containers/0/securityContext/capabilities/add/0", "value":"NET_RAW" }]'
  2. Wait approximately 15 minutes for the pods to fully restart.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB10","label":"Data and AI"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSHGYS","label":"IBM Cloud Pak for Data"},"ARM Category":[{"code":"a8m0z000000GoylAAC","label":"Troubleshooting"}],"ARM Case Number":"","Platform":[{"code":"PF040","label":"Red Hat OpenShift"}],"Version":"4.0.0;4.0.1;4.0.2;4.0.3;4.0.4"}]

Document Information

Modified date:
08 February 2022

UID

ibm16554572

Page Feedback