Adding a remote data source with vaulted credentials fails in Watson Query

When you add or use a data source on a remote agent that is configured with credentials that are stored in a vault, you might see a Failed to update connection details error.

Symptoms

If you use the setRdbcX stored procedure to add or edit a data source connection to a remote data source, you might see an error that is similar to the following message in the logs when you can query the listremotewarnings view.

The exception 'java.sql.SQLException: Failed to update connection details in step 
Request connection details using remote API  call. 
Cause: Failed to retrieve connection details from remote API call.  
Cause: Timeout hit with no response from gateways' was thrown while  evaluating an expression.;

Causes

The resolution of the JDBC details for the remote data source, which includes a call to the vault where the secrets are stored, takes longer than the expected default value of 5 seconds (5000 ms).

Resolving the problem

To work around this issue, you can increase the value of the CONNECTION_DETAILS_REQUEST_TIMEOUT_MS parameter. In the AdminNode log, a log entry that is similar to the following example shows the performance for resolving credentials. This log entry can help you understand the timeout value to set.

2022-01-25 12:24:52.375~823412608 CommonConnUtils ------->  Vault Perf:
        {"ccid":"66aac53d-cfdc-4b30-b525-e4663d4f9cd4","timeunit":"ms","Overall":522,"Catalog":0,
         "Bearer Token":48,"CCS":{"Overall":173,"Api":172,"Parsing":1},"Secrets":301,
         "Detail":[{"secret_urn":"1000330999:MySecret","Api":301,"Parsing":0}]}

Access the AdminNode log.
From navigation menu, click Support > Diagnostics and create a job by selecting the Watson Query checkbox.
Or, run the following commands to access the AdminNode log from a terminal.
1. Log in to the Watson Query head pod.
```
oc rsh c-db2u-dv-db2u-0 bash
```
2. Switch to the Watson Query database instance owner db2inst1.
```
su - db2inst1
```
3. Change to the location of the gaiandb.log file.
```
cd /var/ibm/bigsql/diag/NODE0000/
```
  Note: This log rotates. When it rotates, the name becomes gaiandb#0.log (or 1, 2, 3, ..., n).
To increase the timeout value, use the stored procedure dvsys.setconfigproperty to set the CONNECTION_DETAILS_REQUEST_TIMEOUT_MS configuration parameter to an appropriate value, <timeout value in milliseconds> for <remote node name>. You can use the Run SQL interface to run SQL statements.
```
call dvsys.setconfigproperty('CONNECTION_DETAILS_REQUEST_TIMEOUT_MS','<timeout value in milliseconds>','<remote node name>',?,?
```