Viewing job details, logs, and process reports

Edit online

Use the Jobs page to access job-level information, including source and target connections, job status, execution logs, and detailed process reports. The page helps users to verify job configurations, monitor outcomes, and analyze performance metrics.

Job summary components

IBM® Optim Data Privacy job summaries include the following details:

Component Description
Source The source connection used in the job.
Target The target connection and schema selected.
Configuration Process options and table mappings.
Logs Execution logs and error messages (if any).

The Jobs page provides comprehensive tools for managing and monitoring your job executions. You can perform the following tasks:

Accessing job details and logs

To view the details of an IBM Optim Data Privacy job by using the UI:

  1. From the navigation menu, click Jobs Jobs.
  2. Click a job name to view its details. Alternatively, use the action menu and select View details to access the same information. Use the available options to inspect source, target, and configuration details.
  3. In the logs section:
    • Use View logs to open the standard and error logs, Alternatively, use the action menu in the main page and select View last run log to view the same logs.
    • Use Download logs to download the log report in the JSON format.
  4. To export the logs associated with this job, click Export.
Note: Job status indicators and their meanings:
  • Submitted: The job is submitted to the system and is queued for execution.
  • Running: The job is running and processing data operations.
  • Finished: The job was completed successfully without errors.
  • Failed: The job encountered errors and did not complete successfully. Review the error logs for details.
  • Killed: The job was manually stopped or terminated before completion.
Tip: If a job fails but the Std logs tab does not show an error message, select the Error logs tab. Runtime errors such as table creation failures (OPTTR0222E, CDICO2009E) appear in the Error logs tab only. The Std logs tab contains Apache Spark startup and infrastructure output and does not include job-level runtime errors.

Viewing job process reports

Job process reports provide comprehensive visibility into job execution details and performance metrics. You can view execution statistics, table-level operations, error summaries, and generate PDF reports for offline review and documentation.

To view detailed process reports for a job run:

  1. From the navigation menu, click Jobs Jobs. A list of all jobs are displayed in the table.
  2. From the action menu at the end of a job, click View Reports to open the detailed job process report page. The report contains the following sections:
    Job runs
    Displays a list of all execution runs for the selected job, showing the run ID, execution timestamp, duration, and status for each run. Select a specific run to view its detailed metrics.
    Job summary
    Gives a quick overview of the selected job run, including job ID, start and end times, duration, status, and number of tables processed.
    Row statistics
    Shows aggregate data processing metrics across all tables, including total rows attempted, total rows that are successfully written, total rows failed, and overall success rate. Use these metrics to assess the overall data processing efficiency.
    Table operations
    Presents granular metrics for each table operation, including table name, rows attempted, rows that are written, rows failed, success rate percentage, and operation duration. Status indicators help you quickly identify successful, partially successful, or failed table operations. You can pinpoint specific tables that encountered issues in this section.
    Error summaries
    Lists detailed error information including error codes, error counts, affected tables, and error messages. Use this section to troubleshoot failures, identify data quality issues, and implement preventive measures. Error patterns help you understand recurring problems and their root causes.
  3. Optionally, click Download report to generate and download a comprehensive PDF report for offline review, documentation, or sharing.

Use the insights from job process reports to optimize performance, troubleshoot issues, and implement preventive measures for data quality problems.

Running completed extract and archive jobs

You can re-run completed extract and archive jobs in two ways. The availability and behavior of the Run option depends on whether the target configuration has Allow overwrite enabled. This feature supports execution of the same job configuration without requiring a new job definition, enabling efficient repetition of job executions when needed.

You might need to run the same extract or archive job multiple times due to updated source data, validation activities, testing requirements, or operational reruns. The Run option supports quick reexecution of an existing job, eliminating the need to recreate job configurations and improving operational efficiency.

Important: When you configure a target (such as an archive node in the flow builder), select the Allow overwrite checkbox to enable this node to be reused across multiple flows and write to the same output file.

Running jobs from the Jobs page:

All jobs are listed on the Jobs page, and the overflow menu provides a Run option for every job. When you select Run, the system checks whether the target has Allow overwrite enabled:

  • If Allow overwrite is enabled, the job runs successfully.
  • If Allow overwrite is not enabled, a notification is displayed indicating that the job cannot be run because the target does not have Allow overwrite enabled.

To run a completed extract or archive job from the Jobs page:

  1. From the navigation menu, click Jobs Jobs.
  2. Locate a completed extract or archive job in the jobs table.
  3. From the action menu at the end of the job row, click Run.
  4. If the target has Allow overwrite enabled, the job execution starts immediately, and the job status changes to Running. If not, a notification is displayed.
  5. Monitor the new job execution from the jobs page. The existing target data is overwritten according to the configured overwrite settings.

Running jobs from the View Details page:

You can navigate to the job details through the overflow menu. In this view, the Run option is only displayed for jobs whose target has Allow overwrite enabled.

To run a completed extract or archive job from the View Details page:

  1. From the navigation menu, click Jobs Jobs.
  2. From the action menu at the end of a job row, select View details.
  3. If the target has Allow overwrite enabled, the Run option is displayed. Click Run to start the job execution.
  4. The job execution starts immediately, and the job status changes to Running.
  5. Monitor the new job execution from the jobs page. The existing target data is overwritten according to the configured overwrite settings.

Each job run generates a new execution entry in the job history, enabling tracking of all runs and comparison of results over time. Execution logs and process reports for each run are available for performance analysis and issue troubleshooting.