WKC connector pod in CrashLoopBackoff state

Check the following settings if the Watson™ Connector pod is in the CrashLoopBackoff or Error state.

Check that there are no Watson Knowledge Catalog export policies that are in running state when you perform any changes to Watson Knowledge Catalog connector application configuration.

Check the settings for the following environment variables if you are using IBM Cloud® Watson Knowledge Catalog:
WKC_API_KEY
Check the correct value for the following parameter if you are using IBM Cloud Watson Knowledge Catalog:
WKC_BASE_URI
Check the settings for the following environment variables if you are using an on-premises Watson Knowledge Catalog :
WKC_BASE_URI 
WKC_AUTH_URI
WKC_CONNECTION_MAP 

After successfully exporting records, if you change an environment variable to an invalid value and then export records again, you can end up with unprocessed messages in the Kafka buffer. You must clear these messages before you can change the invalid environment variable to the correct value. Run the following commands to clear the Kafka queue and then restart the Watson Knowledge Catalog connector application.

Clear Queue

/opt/kafka/bin/kafka-topics.sh --delete --zookeeper localhost:2181 --topic WKCConnector_work

Recycle Pod

oc scale deployment --replicas=1 spectrum-discover-wkcconnector

oc scale deployment --replicas=0 spectrum-discover-wkcconnector

If the Watson Knowledge Catalog connector pod remains down after you have performed the above steps, do the following:
  1. When the Watson Knowledge Catalog connector (WKC connector) pod in IBM Spectrum® Discover cannot be started, find the pod name by using the following command:
    kp|grep spectrum-discover-wkcc
    The sample output shows the pod detail similar to the following:
    spectrum-discover spectrum-discover-wkcconnector-<unique_id> 1/1 Running 2d
  2. Issue the following command to retrieve the logs from the WKC connector pod:
    oc logs spectrum-discover-wkcconnector-<unique_id>
  3. If the log terminates with an exception, check the policy engine log for errors by using the following command:
    pollog|grep error
  4. Check if you get the following errors at the time of the exception:
    <time_stamp> - policy.policyapiservice - ERROR - b'Error: Cannot update application, 'WKCConnector', it is in use by the following policies: ["<policy_id>"]. Please stop these policies'
  5. The WKC connector pod does not start until the error shown in step 4 is fixed. Therefore, the running policies must be stopped. Do the following to stop the running policies:
    1. Set up the environmental variables where <sd_password> is the IBM Spectrum Discover sdadmin password.
      
      export SD_USER=sdadmin
      export SD_PASSWORD=<sd_password>
      export OVA=images
    2. Issue the following command to retrieve the authorization token for curl:
      gettoken
    3. Issue the following command to retrieve the policies from IBM Spectrum Discover:
      tcurl -H -k https://<spectrum_discover_url>:443/policyengine/v1/policies | jq
    4. Issue the following command to retrieve the policy history from IBM Spectrum Discover:
      tcurl -H -k https://<spectrum_discover_url>:443/policyengine/v1/policyhistory | jq
    5. Issue the following command to retrieve individual policy history for a given <policy_id>:
      tcurl -H -k https://<spectrum_discover_url>:443/policyengine/v1/policyhistory/<policy_id> | jq
    6. Issue the following command to retrieve log of policy history for a given <history_id>:
      tcurl -H -k https://<spectrum_discover_url>:443/policyengine/v1/policyhistory/<policy_id>/<history_id> | jq
  6. For all the running policies, the <policy_id> for each must be terminated by using the following command:
    tcurl -H -k https://<spectrum_discover_url>:443/policyengine/v1/policies/<policy_id>/kill -X POST |
    jq
  7. Issue the following commands to recycle the pod:
    oc scale deployment --replicas=1 spectrum-discover-wkcconnector
    oc scale deployment --replicas=0 spectrum-discover-wkcconnector