Workflow API (Open Data for Industries)

The Workflow API provides mechanisms to define, manage, and run process definitions in Open Data for Industries.

You can use the Workflow API to:
  • Work with workflow metadata.
  • Run operations for create, retrieve, update, and delete.
  • Trigger workflow in Apache Airflow.
  • Run POST methods to change the status of process startup records.

The Workflow API provides a wrapper around Apache Airflow functions. The API is designed to complete preliminary work with files before you run the Airflow Directed Acyclic Graphs (DAGs) that ingestion data in Open Data for Industries.

To use the File API, see the Workflow API reference.

Learn more

Ingestion workflow

The Workflow service has endpoints to support ingestion of new data files, updating workflow status, delivery of workflow status, and workflow definition.

The ingestion process is critical. The ingestion workflow is initiated by calling the /v1/workflow/{workflow_name}/workflowRun endpoint.

When you call the workflowRun endpoint, the following validation and checks are done:
  1. Validate the incoming request:
    • Verify the authorization token. If the token is missing or is invalid, the ingestion fails and a 401 Unauthorized status response is returned.
    • Verify the partition ID. If the partition ID is missing or is invalid or lacks assigned user groups, the ingestion fails and a 400 Bad Request response is returned.
  2. Submit a new ingestion job to the Open Data for Industries Workflow Engine (Apache Airflow).
  3. Create a workflow data record in the database with a status of submitted.
  4. Return the ID of the workflow to the ingestion service.

The ID that is returned can be used to call other API endpoints for traceability and monitoring.

Workflow API

Keep the following considerations in mind when you use the Workflow API:
  • Each endpoint must receive the authentication bearer token in the Authorization header. Example: 
    Authorization: "Bearer {token}"
  • Each endpoint must receive the partition ID in the Partition-ID header. Example: 
    Partition-Id: "default_partition"
  • The request and response Content Type is application/json

The Workflow API endpoints are illustrated in the following table.

Table 1. Workflow API endpoints
API endpoint Description API reference
GET /v1/workflow List all the workflows for the tenant. Get all workflows
POST /v1/workflow Create a workflow definition. Create or update workflow definition
DELETE /v1/workflow/{workflow_name} Delete a specific workflow based on its name. Delete workflow
GET /v1/workflow/{workflow_name} List the details for a specific workflow based on its name. Get details for a specific workflow
POST /v1/workflow/{workflow_name}/workflowRun Run a specific workflow. Run a workflow
GET /v1/workflow/{workflow_name}/workflowRun List all instances of the specified workflow that were run. Get all run details of a workflow
GET /v1/workflow/{workflow_name}/workflowRun/{runId} List a specific instance of the specified workflow that was run. Get details for a specific run of a workflow
PUT /v1/workflow/{workflow_name}/workflowRun/{runId} Update the workflow of a specific instance. Update the workflow of a specific instance

Workflow API endpoint permissions

The following table describes the Workflow API endpoint permissions.

Table 2. API method permissions
Endpoint URL Method Minimum permissions required
/v1/workflow POST service.workflow.admin
/v1/workflow GET service.workflow.viewer
/v1/workflow/{workflow_name} DELETE service.workflow.admin
/v1/workflow/{workflow_name} GET service.workflow.viewer
/v1/workflow/{workflow_name}/workflowRun POST service.workflow.creator
/v1/workflow/{workflow_name}/workflowRun GET service.workflow.viewer
/v1/workflow/{workflow_name}/workflowRun/{runId} GET service.workflow.viewer
/v1/workflow/{workflow_name}/workflowRun/{runId} PUT service.workflow.viewer