OUA over Kafka JDBC connector version 2.0

Edit online

12.1 and later Diagnose and troubleshoot the OUA issues by using the Kafka JDBC connector version 2.0. Get detailed steps to identify and resolve issues throughout the pipeline, ensuring smooth data flow from the OUA database to the final Guardium report.

Validating data source profiles before installing

  1. Follow the steps from the Validating the collectors status topic to verify that the target collector is online and ready for installation.

  2. Follow the steps from the Testing profile connection topic to verify that the connector details are correct and that the universal connector can connect to the data source.

  3. If step 1 and step 2 validations succeed, install the data source profile.

Installing the data source profile failed

Data source profile installation may fail if the universal connector fails to connect to your data source. Complete the following steps to reconnect the profile.
  1. Follow the steps from the Testing profile connection topic to diagnose the issue.

  2. If the test connection succeeds, follow the steps from Validating the collectors status topic to help ensure that the target collector is ready.

  3. If step 1 and step 2 are successful, install the data source profile again.

Investigating data source profile status issues

Data source profile can display failed connection status such as unsuccessful, or invalid if the collector is offline or is in failed state. Complete the following steps to reconnect the profile.
  1. Follow the steps from the Validating the collectors status topic to verify that the target collector is available.
  2. Re-run the Test Connection to validate the connectivity to the data source.
  3. If the issue persists, uninstall and then install the data source profile.

Data source profile successfully deployed but reports are blank

Verify whether the universal connector is failing to monitor activity from the data source or if data is present but not transmitted to Guardium reports.
  1. Follow the steps from Validate Data in Kafka Topic to help ensure that the data is being monitored by the universal connector.
  2. If the validations in the Validate Data in Kafka topic are successful, follow the steps from the Viewing universal connector health table to confirm that the activity is seen in the universal connector connection.
  3. If activity is seen, help ensure that the correct policy is defined on the collector. For more information, see Configuring policies for Universal Connector.

Troubleshooting steps

Following are the steps to troubleshoot various universal connector issues

Testing profile connection

This test verifies connectivity between the Guardium Kafka cluster and your data source.
  1. In the Data source management page, select one or more OUA profiles and click Test connection.
    • Test connection succeeded: Kafka Connect successfully logged in and queried the data source by using the profile’s details.
    • Test connection failed: Invalid Credentials/Hostname/Query Result: Verify that the connection details are correct and that the data source is operational.
    • Test connection failed: Kafka Error: Go to Kafka Cluster Management page, select the failed cluster, and click Restart. If the issue persists, contact Guardium customer support.

Validating the collectors status

Ensure that the collector is online.
  1. In Guardium Data Protection Central Manager, or Standalone system, click Manage > Universal Connector > Datasource Profile Management.
  2. Select a profile and click Install.
  3. In the Collectors tab, verify that the collector is online (UC status is not must be enabled for OUA over Kafka 2.0).
  4. If the collector is offline, contact Guardium support.

Validating the data in a Kafka topic

Using this section, verify whether the installed universal connector profile is successfully monitoring data from the data source to Guardium® Data Protection by using the Cruise Controls.

  1. To start the Cruise Control, in Guardium Data Protection Central Manager, or Standalone system, click Manage > Universal Connector > Kafka cluster Management.
  2. Select a configured Kafka cluster, click Cruise Control, and then click Start.
  3. Create the Kafka Partition Load Report as follows.
    1. Go to My Dashboard > Create New Dashboard.
    2. Click Add Report and in the filter search Kafka Partition Load.
    3. Click Kafka Partition Load to add the report to the dashboard.
    4. In the report, locate the topic that includes the profile name.
    5. The Message In Rate column to monitor incoming data. This column displays the rate of messages that are being received for that topic.

      If the topic with the profile name exists and messages are visible in the topic, it indicates that UC is successfully pulling data from the data source.

      If no messages are seen or the topic was not found in the report, it means that the universal connector does not received any data from the DB. In that case, verify that database activity is properly logged in the UNIFIED_AUDIT_TRAIL view on the data source side. For more information on cruise control, see Creating Kafka clusters topic.

Viewing universal connector health table

  1. Click Actions > View UC health > Deployment Health Table.
  2. The S-TAPs tab provides an overview of UC connections and their traffic status. The connection name must contain the profile name.
Read the connection status as follows.
  • Active Status: The connection displays in the S-TAPs tab as active, indicating that universal connector is successfully capturing traffic. This is an ideal scenario.
  • Inactive Status (Previously Active): The connection displays as inactive, which means that the profile was previously active and received data, but traffic is not available from the past hour. This is configurable by using the set_universal_connector_data_timeout API.
  • Inactive Status (No Data Observed): The connection displays as inactive and was never active before, indicating that the universal connector has not observed any data. This suggests a potential issue that requires further investigation. Contact Guardium support.
For more information on health tables, see Deployment health topology and table views.

Runtime error during high-rate data auditing

When auditing data at high rates (12,000 rows per second), the system might experience performance degradation or runtime errors due to Oracle database constraints. These errors occur when the volume of audit records exceeds the processing capacity of the temporary tablespace (TEMP).

The following ORA-01652 error is displayed when query operations on the audit table, such as large sorts, joins or aggregations, require more temporary space than what is available in the TEMP tablespace.
ORA-01652 error: unable to extend temp segment by <N> in tablespace TEMP.
You can use any one of the following methods to troubleshoot the ORA-01652 error:
  • Update the audit policy to reduce the volume of generated audit records.
  • Increase the size of the TEMP tablespace to accommodate higher loads.