schemes: - http - https swagger: "2.0" info: description: | IBM Optim core functions and application management ### Security - Every request must include the `Authorization` HTTP header with the value `Bearer [access_token]`. - Most API request optim-conn-profile in header to call for database. `optim-conn-profile: [Connection profile name]` license: name: (c) IBM. All rights reserved title: IBM Optim Archive - Core API server version: 1.2.0 basePath: /optim paths: /v1/auth/logout: post: consumes: - application/x-www-form-urlencoded description: Logout the user. operationId: createAuthLogout parameters: - in: formData name: client_id type: string - in: formData name: refresh_token required: true type: string produces: - application/json responses: "204": description: Successfully logged out. No content returned. "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid refresh token message: Invalid token. Please check your refresh_token messageID: OPTTC0507E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: Invalid or expired refresh token message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: An unexpected error occurred while processing your request message: 'Unexpected error: Request processing failed' messageID: OPTTC0512E schema: $ref: '#/definitions/errorResponse' default: description: Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: An unexpected error occurred while processing your request message: 'Unexpected error: Request processing failed' messageID: OPTTC0512E schema: $ref: '#/definitions/errorResponse' summary: Auth Logout tags: - Auth /v1/auth/token: post: consumes: - application/x-www-form-urlencoded description: | Create a auth token. operationId: createAuthToken parameters: - default: password in: formData name: grant_type type: string - in: formData name: client_id type: string - in: formData name: username required: true type: string - in: formData name: password required: true type: string - default: openid organization in: formData name: scope type: string produces: - application/json responses: "201": description: Auth Token created examples: application/json: _statusCode_: !!float 201 message: Auth token retrieved successfully messageID: OPTTC0501I requestObj: access_token: ee6tOO1xgafAOqCWfl4WJq7ExR-CrirRhVzTPLryAwZp14uLwvz0vT_8wJpQ9idf3TfnnPDHDJhj1GfB8j3k0zxvnXMcZjtvZwfmog expires_in_sec: !!float 900 refresh_expires_in_sec: !!float 1800 refresh_token: eyJhbGciOiJIUzUxMiIsInOTczMzFjN2YwIxMDk0NGRhZCJ9.vUje0IfZZcP9Q6GK0vT0Kz1LUl9wW9vu4LXeN5RaJYepwzdsdsf schema: $ref: '#/definitions/successAuthTokenResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request: required OAuth fields missing or invalid' message: 'Core received a bad request: grant_type, username, or password not provided' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: Invalid user credentials message: Invalid credentials. Please check your username and password messageID: OPTTC0504E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User does not have required permissions message: 'Forbidden: Access denied' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: An unexpected error occurred while processing your request message: 'Unexpected error: Request processing failed' messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: An unexpected error occurred while processing your request message: 'Unexpected error: Request processing failed' messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' summary: Create a auth token tags: - Auth /v1/auth/token/refresh: post: consumes: - application/x-www-form-urlencoded description: | Create a auth refresh token. operationId: createAuthRefreshToken parameters: - in: formData name: client_id type: string - in: formData name: refresh_token required: true type: string produces: - application/json responses: "200": description: Auth refresh token created examples: application/json: _statusCode_: !!float 200 message: Auth token retrieved successfully messageID: OPTTC0501I requestObj: access_token: ee6tOO1xgafAOqCWfl4WJq7ExR-CrirRhVzTPLryAwZp14uLwvz0vT_8wJpQ9idf3TfnnPDHDJhj1GfB8j3k0zxvnXMcZjtvZwfmog expires_in_sec: !!float 900 refresh_expires_in_sec: !!float 1800 refresh_token: eyJhbGciOiJIUzUxMiIsInOTczMzFjN2YwIxMDk0NGRhZCJ9.vUje0IfZZcP9Q6GK0vT0Kz1LUl9wW9vu4LXeN5RaJYepwzdsdsf schema: $ref: '#/definitions/successAuthRefreshTokenResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid refresh token message: Invalid token. Please check your refresh_token messageID: OPTTC0507E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: Token validation failed message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: An unexpected error occurred while processing your request message: 'Unexpected error: Request processing failed' messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: An unexpected error occurred while processing your request message: 'Unexpected error: Request processing failed' messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Create a auth refresh token tags: - Auth /v1/auth/userinfo: get: description: Retrieves user information for the authenticated user. operationId: getAuthUserinfo parameters: - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string produces: - application/json responses: "200": description: Successfully retrieved user info examples: application/json: _statusCode_: !!float 200 message: Userinfo retrieved successfully messageID: OPTTC0510I requestObj: email: user1@example.com family_name: Test given_name: User1 name: User1 Test preferred_username: user1 sub: 49df-b41a-4c98b35acce0 schema: $ref: '#/definitions/getAuthUserinfoSuccessResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: An unexpected error occurred while processing your request message: 'Unexpected error: Request processing failed' messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' default: description: Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: An unexpected error occurred while processing your request message: 'Unexpected error: Request processing failed' messageID: OPTTC0511E schema: $ref: '#/definitions/errorResponse' summary: Retrieve user info tags: - Auth /v1/compare/submit: post: consumes: - application/vnd+ibm.dataplatform-pipeline+json description: | Submit a compare operation for execution operationId: submitCompare parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Compare pipeline-flow-v3 payload in: body name: body required: true schema: $ref: '#/definitions/comparePayload' produces: - application/json responses: "201": description: Compare submission successful schema: $ref: '#/definitions/compareResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 message: 'Validation error occurred: Invalid compare payload structure' messageID: OPTTC0400E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0401E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 message: 'Forbidden: You do not have permission to submit compare operations' messageID: OPTTC0403E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 message: 'Unexpected error: An internal server error occurred while submitting compare operation' messageID: OPTTC0500E schema: $ref: '#/definitions/errorResponse' default: description: Error schema: $ref: '#/definitions/errorResponse' summary: Submit compare job tags: - Compare /v1/config: get: description: Get product configurations from environment variables operationId: getConfig responses: "200": description: Fetched environment variables examples: application/json: _statusCode_: !!float 200 message: Configuration fetched from environment variables successfully messageID: OPTTC0010I requestObj: is_product_archive: true is_product_dp: false is_product_tdm: false license_info: archive: licence_type: enterprise product_description: ' ' product_name: Data Archive product_version: 1.2.0 release_date: ' ' ui_cache_enabled: "" ui_cache_expiration_in_millis: "" schema: $ref: '#/definitions/getConfigSuccessResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 message: 'Validation error occurred: Invalid request parameters' messageID: OPTTC0006E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 message: 'Forbidden: You do not have permission to access this resource' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 message: 'Unexpected error: An internal server error occurred' messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error examples: application/json: _statusCode_: !!float 500 message: 'Unexpected error: An internal server error occurred' messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get product configurations from environment variables tags: - Config /v1/connprofiles: get: description: Administrators can retrieve the list of all connection profiles in the system. Regular users will receive a list to which they have accessibility operationId: getAllConnectionProfiles parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string produces: - application/json responses: "200": description: The basic properties of all connection profiles examples: application/json: _statusCode_: !!float 200 message: Retrieved all connection profiles. messageID: OPTTC0211I requestObj: infos: - connection: name: my-postgres-db properties: database: mydb host: db.example.com password: '{archonenc}oAntycmZV0IuLEeRqPE60Dl4OHy2bS4QqwS3lMecRxzTt7ZryjdJf0EAbU4=' port: "5432" ssl: "true" type: postgresql username: admin ref: 123e4567-e89b-12d3-a456-426614174000 last_updated: "2025-11-17T14:47:28.08535Z" user_id: 49ef049d-7a8d-4edf-b41a-4c98b35acce0 user_name: User1 Test1 schema: $ref: '#/definitions/getAllConnectionProfilesSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid UUID message: 'Core received a bad request: Invalid UUID' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: invalid token message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: my-postgres-db' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Retrieve all connection profiles tags: - Connection Profile post: consumes: - application/json description: "Create a new database connections and storage profiles. Both SSL and non-SSL connections can be created.\n**Note:** SSL certificate, if provided, must be a **base64 encoded** string.\n\n---\n\n## Database-Specific Required Fields\n**Supported database/storage type identifier values:** `db2, db2zos, informix, localfs, mongodb, netezza, oracle, postgresql, snowflake, sqlserver, sybase, hive, mysql, teradata, amazons3, azureblobstorage`\n\n### 1. **PostgreSQL, IBM Db2, IBM Db2 for z/OS, Microsoft SQL Server, Oracle, Netezza, Sybase, Apache Hive, MySQL**\n**Required Fields:**\n- `type` (string): Database type identifier\n- `database` (string): Database name\n- `host` (string): Database host address\n- `port` (string): Database port number\n- `username` (string): Database username\n- `password` (string): Database password\n- `ssl` (string): SSL enabled flag - must be \"true\" or \"false\"\n - If `ssl` = \"true\", then `ssl_certificate` (string) is also required\n\n---\n\n### 2. **Snowflake**\n**Required Fields:**\n- `type` (string): Database type identifier\n- `account_name` (string): Snowflake account identifier\n- `warehouse` (string): Snowflake warehouse name\n- `database` (string): Database name\n- `host` (string): Snowflake host address\n- `username` (string): Username\n- `password` (string): Password\n\n**Note:** SSL certificate is NOT required even if SSL is enabled\n\n---\n\n### 3. **MongoDB**\n**Required Fields:**\n- `type` (string): Database type identifier\n- `database` (string): Database name\n- `host` (string): MongoDB host address\n- `port` (string): MongoDB port number\n- `username` (string): Username\n- `password` (string): Password\n- `auth_database` (string): Authentication database name\n- `ssl` (string): SSL enabled flag - must be \"true\" or \"false\"\n - If `ssl` = \"true\", then `ssl_certificate` (string) is also required\n\n---\n\n### 4. **Teradata**\n**Required Fields:**\n- `type` (string): Database type identifier\n- `authentication_method` (string): Authentication method\n- `host` (string): Teradata host address\n- `username` (string): Username\n- `password` (string): Password\n\n**Note:** SSL certificate is NOT required even if SSL is enabled\n\n---\n\n### 5. **IBM Informix**\n**Required Fields:**\n- `type` (string): Database type identifier\n- `server` (string): Informix server name\n- `host` (string): Host address\n- `port` (string): Port number\n- `username` (string): Username\n- `password` (string): Password\n- `ssl` (string): SSL enabled flag - must be \"true\" or \"false\"\n - If `ssl` = \"true\", then `ssl_certificate` (string) is also required\n\n---\n\n### 6. **Amazon S3**\n**Required Fields:**\n- `type` (string): Database type identifier\n- `auth_method` (string): Authentication method`\n\nIf `auth_method` = `basic_credentials`\n - `access_key` (string): Access key\n - `secret_key` (string): Secret key\n\nIf `auth_method` = `temporary_credentials`\n - `access_key` (string): Access key\n - `secret_key` (string): Secret key\n - `session_token` (string): Session token\n\nIf `auth_method` = `trusted_role_credentials` \n - `access_key` (string): Access key\n - `secret_key` (string): Secret key\n - `role_arn` (string): Role ARN\n - `role_session_name` (string): Role session name\n\n---\n\n### 7. **Azure Blob Storage**\n**Required Fields:**\n- `type` (string): Database type identifier\n- `auth_method` (string): Authentication method`\n\nIf `auth_method` = `connection_string`\n - `connection_string` (string): Connection string`\n\nIf `auth_method` = `entra_id\n - `tenant_id` (string): Tenant ID\n - `client_id` (string): Client ID\n - `client_secret` (string): Client secret\n - `storage_account_url` (string): Storage account URL\n\nIf `auth_method` = `entra_id_user` \n - `client_id` (string): Client ID\n - `username` (string): Username\n - `password` (string): Password\n - `storage_account_url` (string): Storage account URL\n\n---\n\n### 8. **Local File System**\n**Required Fields:**\n- `type` (string): Database type identifier\n- `file_type` (string): Type of file system\n" operationId: createConnectionProfile parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: All required connection profile properties for creation in: body name: body required: true schema: $ref: '#/definitions/connectionProfile' produces: - application/json responses: "201": description: Connection profile created examples: application/json: _statusCode_: !!float 201 id: 3c2ff60f-60f6-eaf5a33b45cf message: 'Connection profile created: my_postgres_db' messageID: OPTTC0200I schema: $ref: '#/definitions/successPostResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 errors: [] exception: password property is empty or not a string message: 'Core received a bad request: invalid connection properties' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: invalid token message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: my-postgres-db' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "409": description: Connection profile already existed examples: application/json: _statusCode_: !!float 409 errors: [] exception: record already exists message: 'Profile already exists: my_postgres_db' messageID: OPTTC0210E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Create a new connection profile tags: - Connection Profile /v1/connprofiles/testConnection: post: consumes: - application/json description: | Validates a given connection profile by attempting to connect to the database. operationId: testConnection parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: "Connection profile details, including database type, host, port, \nusername, password, SSL configuration, etc.\n" in: body name: body required: true schema: $ref: '#/definitions/connectionProfile' produces: - application/json responses: "200": description: Successful connection. Database is reachable with the given profile. examples: application/json: _statusCode_: !!float 200 message: Connection to the data source was successful. messageID: OPTTC0208I schema: $ref: '#/definitions/successResponse' "400": description: Bad request. Input validation failed (e.g. missing host, invalid type). examples: application/json: _statusCode_: !!float 400 errors: [] exception: password property is empty or not a string message: 'Core received a bad request: invalid connection properties' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized. The Authorization header is missing or invalid. examples: application/json: _statusCode_: !!float 401 errors: [] exception: invalid token message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden. User does not have permission to access this connection profile. examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: my_postgres' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error. An unexpected error occurred while testing the connection. examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload in case of unhandled errors. examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Check whether the database can be connected tags: - Connection Profile /v1/connprofiles/{name}: delete: description: | Deletes an existing connection profile by name. Administrators can delete any profile in the system. Regular users can only delete profiles they own or have been granted access to. operationId: deleteConnectionProfile parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: The specific name of the connection profile which the session user wants to delete. in: path name: name required: true type: string x-example: my_postgres_db produces: - application/json responses: "200": description: Connection profile delete successfully examples: application/json: _statusCode_: !!float 200 message: 'Successfully deleted connection profile: my_postgres_db' messageID: OPTTC0206I schema: $ref: '#/definitions/successResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid UUID message: 'Core received a bad request: Invalid UUID' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: invalid token message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Removal of this connection profile is not allowed examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: my_postgres' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: The connection profile does not exist examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Profile not found: my_postgres' messageID: OPTTC0213E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Delete existing connection profile tags: - Connection Profile get: description: "Retrieve the properties of a connection profile by name. \nAdministrators can retrieve properties of all profiles. Regular users can only retrieve profiles they own or have been granted access to.\n" operationId: getConnectionProfileByName parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: The specific name of the connection profile which the session user wants to retrieve. in: path name: name required: true type: string x-example: my_postgres_db produces: - application/json responses: "200": description: The basic properties of a connection profile examples: application/json: _statusCode_: !!float 200 message: 'Fetched connection profile: my-postgres-db' messageID: OPTTC0202I requestObj: connection: name: my-postgres-db properties: database: mydb description: Production Postgres database host: db.example.com password: '{archonenc}oAntycmZV0IuLEeRqPE60Dl4OHy2bS4QqwS3lMecRxzTt7ZryjdJf0EAbU4=' port: "5432" ssl: "true" type: postgresql username: dbuser ref: 123e4567-e89b-12d3-a456-426614174000 last_updated: "2025-11-19T09:10:45.873174Z" user_id: 49ef049d-7a8d-4edf-b41a-4c98b35acce0 user_name: User1 Test1 schema: $ref: '#/definitions/getConnectionProfileInfoSuccessResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid UUID message: 'Core received a bad request: Invalid UUID' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: invalid token message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: my_postgres' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Profile not found: my_postgres' messageID: OPTTC0213E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Retrieve the specific connection profile tags: - Connection Profile put: consumes: - application/json description: | Updates a specific connection profile by name. Administrators can update any profile in the system. Regular users can only update profiles they own or have been granted access to. See the **connectionProfile** model for full details on all properties. operationId: updateConnectionProfile parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: The specific name of the connection profile which the session user wants to update. in: path name: name required: true type: string x-example: my_postgres_db - description: The connection profile properties in: body name: body required: true schema: $ref: '#/definitions/connectionProfile' produces: - application/json responses: "200": description: Update the specific connection profile successfully examples: application/json: _statusCode_: !!float 200 message: 'Successfully updated connection profile: postgres_source_db' messageID: OPTTC0204I requestObj: connection: name: my-postgres-db properties: database: mydb description: Production Postgres database host: db.example.com port: "5432" ssl: "true" type: postgres username: dbuser ref: 123e4567-e89b-12d3-a456-426614174000 last_updated: "2025-07-21T09:51:41.324+05:30" user_id: 885f509-ea59-43fc-b1ad-f050 user_name: User2 Test2 schema: $ref: '#/definitions/getConnectionProfileInfoSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: password property is empty or not a string message: 'Core received a bad request: invalid connection properties' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: invalid token message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: my_postgres' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: The connection profile does not exist examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Profile not found: my_postgres' messageID: OPTTC0213E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Update the specific connection profile tags: - Connection Profile /v1/connprofiles/{name}/dependencies: get: description: | Retrieve dependent nodes and flows for a specific connection profile. Administrators can retrieve dependencies of all profiles. Regular users can only retrieve dependencies for profiles they own or have been granted access to. operationId: getConnectionProfileDependencies parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Connection profile name in: path name: name required: true type: string produces: - application/json responses: "200": description: List of flows and nodes associated to a connection profile examples: application/json: _statusCode_: !!float 200 message: 'Connection profile dependencies fetched: postgres-connection' messageID: OPTTC0214E requestObj: flows: - id: 4b42e250-fff3-48f2-97b9-a7aeaf147544 name: postgres-flow-1 - id: 1365e2ba-ca12-440b-8a17-42fcec65fd6e name: postgres-flow-2 nodes: access_definition: - id: 2660598d-5663-4c54-8389-d42fd8d5ae5d name: Postgres-access insert_definition: - id: b46611c7-9f8e-4bee-8950-e8495e615b12 name: insert-1 - id: b81bbbf4-b7ec-469f-9a01-cbaa6ff0acb2 name: insert-2 - id: 7cbaeb1b-8575-40f6-865f-e5890d2f13dd name: insert-3 subset_definition: - id: cce3892d-8231-4c7a-8dbf-46fa38643fdd name: subset-1 - id: be80545e-1ea3-4467-9781-a1eb8e172e1c name: subset-2 schema: $ref: '#/definitions/getConnectionDependenciesSuccessResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid UUID message: 'Core received a bad request: Invalid UUID' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: invalid token message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: postgres-connection' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Profile not found: postgres-connection' messageID: OPTTC0213E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: database connection error message: 'Error fetching connection profile dependencies: postgres-connection' messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Retrieve dependent nodes and flows for a specific connection profile tags: - Connection Profile /v1/database/schemas: get: consumes: - application/json description: "Retrieves schemas from database based on the provided connection profile. \nBy default, results are returned in alphabetical order by schema name. \nSupports pagination, search, and filtering of system schemas.\n" operationId: listSchemas parameters: - description: The connection profile name that identifies which database to query. in: header name: optim-conn-profile required: true type: string x-example: my_postgres_db - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Optional search term to filter schemas by name. in: query name: search_name type: string - default: !!float 0 description: Number of items to skip before starting to return results (for pagination). in: query name: offset type: integer - default: !!float 100 description: Maximum number of schemas to return in the response. in: query name: limit type: integer - description: Whether to include system schemas in the results. enum: - "true" - "false" in: query name: show_systems type: string - description: Sort order for schema names. Use 'true' for ascending and 'false' for descending. enum: - "true" - "false" in: query name: is_ascend type: string produces: - application/json responses: "200": description: list of schemas examples: application/json: _statusCode_: !!float 200 message: Successfully retrieved schemas messageID: OPTTC0400I requestObj: resources: - definer_type: U name: public - definer_type: U name: testschema1 total_count: !!float 2 schema: $ref: '#/definitions/getSchemasSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: my_postgres' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: List schemas in the database tags: - Database Objects /v1/database/schemas/{schema_name}: get: description: "Retrieves detailed information about a specific schema in the connected database. \nThe schema is identified by its name, provided in the path parameter.\n" operationId: getSchema parameters: - description: The connection profile name that identifies which database to query. in: header name: optim-conn-profile required: true type: string x-example: my_postgres_db - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: The name of the schema to retrieve. in: path name: schema_name required: true type: string x-example: public responses: "200": description: Schema information examples: application/json: _statusCode_: !!float 200 message: Successfully retrieved schema public from connection profile my_postgres_db messageID: OPTTC0408I requestObj: definer_type: U name: public schema: $ref: '#/definitions/getSchemaSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: my_postgres' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Schema not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: 'com.ibm.dataarchon.runtime.exception.SchemaNotFoundException: public' message: 'Failed to retrieve schema info: com.ibm.dataarchon.runtime.exception.SchemaNotFoundException: public' messageID: OPTTC00404E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get schema details tags: - Database Objects /v1/database/schemas/{schema_name}/tables: get: description: "Retrieves all tables in the specified schema of the connected database. \nThe results are returned alphabetically by table name. Supports pagination, search filtering, and optional inclusion of system tables.\n" operationId: listSchemaTables parameters: - description: Connection profile name in: header name: optim-conn-profile required: true type: string x-example: my_postgres_db - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Schema name in: path name: schema_name required: true type: string x-example: public - description: name to match in: query name: search_name type: string - default: !!float 0 description: The number of items to skip before starting to collect the result set in: query name: offset type: integer - default: !!float 100 description: The number of items to return in: query name: limit type: integer - description: sort name in ascending order enum: - "true" - "false" in: query name: is_ascend type: string produces: - application/json responses: "200": description: List of tables examples: application/json: _statusCode_: !!float 200 message: Successfully listed schema tables messageID: OPTTC0406I requestObj: resources: - name: customers schema: public - name: orders schema: public total_count: !!float 2 schema: $ref: '#/definitions/getSchemaTablesSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: my_postgres' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Database or schema not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Profile not found: my_postgres_db' messageID: OPTTC0208E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Lists tables in a schema tags: - Database Objects /v1/database/schemas/{schema_name}/tables/{table_name}: get: description: "Retrieves detailed information about a specific table within a schema in the connected database. \nThe table is identified by its name in the path parameter.\n" operationId: getSchemaTable parameters: - description: Connection profile name in: header name: optim-conn-profile required: true type: string x-example: my_postgres_db - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Schema name in: path name: schema_name required: true type: string x-example: public - description: Table name in: path name: table_name required: true type: string - description: Optional access definition ID to filter columns by access definition. in: query name: access_definition_id type: string produces: - application/json responses: "200": description: Table information examples: application/json: _statusCode_: !!float 200 message: Successfully retrieved table definition messageID: OPTTC0402I requestObj: columns: - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: first_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: last_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: email nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 100 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: phone nullable: yes type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 20 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: address nullable: yes type_modifier: TEXT type_name: LONGVARCHAR type_precision: !!float 1073741823 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: credit_card nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 19 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: credit_card_type nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 30 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: birth_date nullable: no type_modifier: DATE type_name: DATE type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: created_at nullable: yes type_modifier: TIMESTAMP type_name: TIMESTAMP type_precision: !!float 26 type_scale: !!float 6 type_string_codeunit: "" comments: PostgreSQL is an open source and customizable object-relational database. For the PostgreSQL connector on IBM Cloud, select IBM Cloud Databases for PostgreSQL. conn_profile_name: postgres_db name: customers pk_info: columns: - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" comments: Primary key constraint conn_profile_name: postgres_db name: customers_pkey schema_name: public table_name: customers schema: public stats: last_collected: "2025-11-20T17:47:43.496Z" rows: !!float 0 schema: $ref: '#/definitions/getSchemaTableByNameSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: my_postgres' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Table not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Profile not found: my_postgres_db' messageID: OPTTC0208E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Returns information about a table tags: - Database Objects /v1/database/schemas/{schema_name}/tables/{table_name}/data: get: description: "Fetches data from a specific table, up to a maximum of 100,000 rows. \nCurrently, tables containing CLOB, BLOB, or DBCLOB columns cannot be retrieved. \nSupports pagination using `offset` and `limit` query parameters.\n" operationId: getTableData parameters: - description: Connection profile name in: header name: optim-conn-profile required: true type: string x-example: my_postgres_db - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Schema name in: path name: schema_name required: true type: string x-example: public - description: Table name in: path name: table_name required: true type: string x-example: my_table - default: !!float 0 description: The number of rows to skip before starting to collect the result set in: query name: offset type: integer - default: !!float 100 description: The number of rows to return in: query name: limit type: integer produces: - application/json responses: "200": description: Returned result was successful examples: application/json: _statusCode_: !!float 200 message: Successfully retrieved table data messageID: OPTTC0404I requestObj: columns: - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: first_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: last_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: email nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 100 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: phone nullable: yes type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 20 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: address nullable: yes type_modifier: TEXT type_name: LONGVARCHAR type_precision: !!float 1073741823 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: credit_card nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 19 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: credit_card_type nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 30 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: birth_date nullable: no type_modifier: DATE type_name: DATE type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: created_at nullable: yes type_modifier: TIMESTAMP type_name: TIMESTAMP type_precision: !!float 26 type_scale: !!float 6 type_string_codeunit: "" rows: - - "1" - Alice - Johnson - alice@example.com - 123-456-7890 - 123 Maple St - 4111-1111-1111-1111 - visa - "1985-03-12" - "2025-06-05 21:20:34.022267" - - "2" - Bob - Smith - bob@example.com - 987-654-3210 - 456 Oak St - 5500-0000-0000-0004 - visa - "1992-07-25" - "2025-06-05 21:20:34.022267" - - "3" - Charlie - Davis - charlie@example.com - 555-666-7777 - 789 Pine St - 4532-6993-8458-6192 - visa - "1988-05-30" - "2025-06-05 21:20:34.022267" total_count: !!float 20 schema: $ref: '#/definitions/getTableDataSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: my_postgres' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Table not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Profile not found: my_postgres_db' messageID: OPTTC0208E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get table data tags: - Database Objects /v1/database/schemas/{schema_name}/tables/{table_name}/dependencies: get: description: "Retrieves dependency information for a specific table, including ancestor tables it depends on and/or descendant tables that depend on it. \nResults are returned in order from parent to child. Supports optional depth and direction parameters.\n" operationId: getTableDependencies parameters: - description: Connection profile name in: header name: optim-conn-profile required: true type: string x-example: my_postgres_db - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Schema name in: path name: schema_name required: true type: string x-example: public - description: The start table name in: path name: table_name required: true type: string x-example: my_table - default: ancestors description: direction to fetch ancestors, descendants or both enum: - ancestors - descendants - both in: query name: direction type: string - default: !!float 10 description: The number of levels of dependencies to retrieve in: query name: depth type: integer produces: - application/json responses: "200": description: Table dependencies info examples: application/json: _statusCode_: !!float 200 message: Fetched for dependencies for table public.orders messageID: OPTTC0410I requestObj: ancestors: - name: customers schema: public conn_profile_name: my_postgres_db depth: !!float 10 descendants: - name: order_items schema: public direction: both related_tables: - name: customers schema: public - name: order_items schema: public schema_name: public table_name: orders schema: $ref: '#/definitions/getTableDependenciesSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: my_postgres' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Table not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Profile not found: my_postgres_db' messageID: OPTTC0208E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get table dependencies tags: - Database Objects /v1/extract/job: delete: consumes: - application/json description: | Delete one or more extract jobs by their job IDs. This endpoint enables users to remove extract files when they are no longer needed. Supports batch deletion of multiple extract jobs in a single request. operationId: deleteExtractJob parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - default: extract description: Operation type for the extract enum: - extract in: query name: operation_type type: string x-example: extract - description: Job IDs to be delete in: body name: body required: true schema: $ref: '#/definitions/deleteResourceArray' produces: - application/json responses: "200": description: Job deleted examples: application/json: _statusCode_: !!float 200 message: 'Successfully deleted extract job(s): 0dfb8b8b-fc93-42d1-b193-3d7d8c92b861' messageID: OPTTC0904I schema: $ref: '#/definitions/successResponse' "207": description: Multiple Status examples: application/json: _statusCode_: !!float 207 errors: - id: 0dfb8b8b-fc93-3d7d8c92b86c reason: record not found status: failed_to_delete - id: 0dfb8b8b-fc93-3d7d8c92b863 reason: record not found status: failed_to_delete exception: 'partial_delete_error: one or more job IDs could not be deleted due to missing or invalid references' message: 'Failed to delete extract job(s), Error: Some jobs could not be deleted' messageID: OPTTC0904E schema: $ref: '#/definitions/errorResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 403 errors: [] exception: Extract file not found message: Resource not found. messageID: OPTTC0904E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Delete extract Job tags: - Extract get: description: | Retrieve detailed information about a specific extract job using its job ID. Users can view comprehensive details about completed extract operations and navigate to associated job details. Essential for tracking and managing extract file lifecycle within the application. operationId: getExtractJob parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: The id of the extract to retrieve. in: query name: id required: true type: string - default: extract description: Operation type for the extract enum: - extract in: query name: operation_type type: string x-example: extract produces: - application/json responses: "200": description: extracts retrieved successfully examples: application/json: _statusCode_: !!float 200 message: 'Extract job retrieved successfully, jobID: 0dfb8b8b-fc93-42d1-b193-3d7d8c92b861' messageID: OPTTC0902I schema: $ref: '#/definitions/extractJobSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch extract message: 'Forbidden: my-postgres-db' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get extract by jobID tags: - Extract /v1/extract/savedlist: get: description: | Retrieve a comprehensive list of all saved extract flows with their associated metadata. Users can view all their extract files in one centralized location, making it easy to track extraction history and manage extract file lifecycle. Essential for the Extract Files feature that provides visibility into all extraction operations. operationId: listExtractFlows parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string produces: - application/json responses: "200": description: extracts retrieved successfully examples: application/json: _statusCode_: !!float 200 message: Successfully listed saved extracts messageID: OPTTC0911I requestObj: resources: - access_definition_name: access_node created_at: "2025-11-17T14:53:20.582834Z" created_by: 49ef049d-7a8d-4edf-b41a-4c98b35acce0 created_by_name: User1 Test flow_id: ff11a94f-2a8b-4b88-83d4-2c35ef0222b7 flow_name: extract_flow_001 flow_revision: "2" id: 054750db-8dac-4218-a06d-742b84ed46e2 job_id: 9186134c-8431-4e4a-88b2-1f3555d9a032 name: extract_001 status: FINISHED submission_id: driver-20251117145320-0000 total_count: !!float 1 schema: $ref: '#/definitions/listExtractFlowsSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch extract message: 'Forbidden: my-postgres-db' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: List saved extract flows tags: - Extract /v1/extract/schemas: get: description: | Retrieve the schema structure and table list from a specific extract flow. This endpoint provides an overview of all schemas within an extract file, enabling users to browse the hierarchical structure of extracted data. Returns list of schema names. operationId: getExtractSchemas parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: The id of the flow to retrieve. in: query name: id required: true type: string - default: extract description: Operation type for the extract enum: - extract in: query name: operation_type type: string x-example: extract produces: - application/json responses: "200": description: List extract schemas success examples: application/json: _statusCode_: !!float 200 message: Successfully retrieved extract schemas messageID: OPTTC0905I requestObj: schemas: - name: public tables: - name: customers - name: email_address - name: orders schema: $ref: '#/definitions/getExtractSchemasResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch extract message: 'Forbidden: my-postgres-db' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: extract flow not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Flow details not found, flowID: 0dfb8b8b-d7d8c-92b861' messageID: OPTTC0322E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get extract schemas tags: - Extract /v1/extract/schemas/{schema_name}/tables/{table_name}: get: description: | Retrieve detailed information about a specific table within an extract file. Examine individual table columns. The response includes comprehensive metadata that helps validate extraction results and understand the structure of extracted data. Supports both encrypted and unencrypted extract files. operationId: getExtractTableDetails parameters: - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string - description: Schema name in: path name: schema_name required: true type: string x-example: dbo - description: Table name in: path name: table_name required: true type: string x-example: customers - description: Job ID associated with the extract schema in: query name: id required: true type: string - default: extract description: Operation type for the extract enum: - extract in: query name: operation_type type: string x-example: extract - default: !!float 0 description: The number of items to skip before starting to collect the result set in: query name: offset type: integer - default: !!float 100 description: The number of items to return in: query name: limit type: integer produces: - application/json responses: "200": description: Table information examples: application/json: _statusCode_: !!float 200 message: 'Getting extract table details for job:7cd71244-aac0-4abc-91f6-929d7c19fa18, schema: public, table: customers' messageID: OPTTC0901I requestObj: columns: - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: first_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: last_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: email nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 100 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: phone nullable: yes type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 20 type_scale: !!float 0 type_string_codeunit: "" comments: PostgreSQL is an open source and customizable object-relational database. For the PostgreSQL connector on IBM Cloud, select IBM Cloud Databases for PostgreSQL. conn_profile_name: postgres_db name: customers pk_info: columns: - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" comments: Primary key constraint conn_profile_name: postgres_db name: customers_pkey schema_name: public table_name: customers schema: public stats: last_collected: "2025-11-20T17:47:43.496Z" rows: !!float 0 schema: $ref: '#/definitions/getExtractTableDetailSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection proextract message: 'Forbidden: conn_name' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Table not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'extract table not found: customers' messageID: OPTTC0208E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: get Extract Table Details tags: - Extract /v1/extract/schemas/{schema_name}/tables/{table_name}/data: get: description: | Retrieve sample data rows from a specific table within an extract file. This endpoint enables users to browse and validate actual extracted data by viewing table contents with limit,offset and other query params support. Returns column definitions along with data rows, allowing users to verify extraction accuracy without needing external tools. Supports offset and limit parameters for efficient data browsing. Works with both encrypted and unencrypted extract files. operationId: getExtractTableData parameters: - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string - description: Schema name in: path name: schema_name required: true type: string x-example: dbo - description: Table name in: path name: table_name required: true type: string x-example: customers - description: Job ID associated with the extract schema in: query name: id required: true type: string x-example: 7cd71244-aac0-4abc-91f6-929d7c19fa18 - default: extract description: Operation type for the extract enum: - extract in: query name: operation_type type: string x-example: extract - default: !!float 0 description: The number of items to skip before starting to collect the result set in: query name: offset type: integer - default: !!float 100 description: The number of items to return in: query name: limit type: integer produces: - application/json responses: "200": description: Table information examples: application/json: _statusCode_: !!float 200 message: 'Getting extract table data for job:7cd71244-aac0-4abc-91f6-929d7c19fa18, schema: dbo, table: customers' messageID: OPTTC0903I requestObj: columns: - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: first_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: last_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: email nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 100 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: phone nullable: yes type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 20 type_scale: !!float 0 type_string_codeunit: "" rows: - - "1" - Alice - Johnson - alice@example.com - 123-456-7890 - - "2" - Bob - Smith - bob@example.com - 987-654-3210 - - "3" - Charlie - Davis - charlie@example.com - 555-666-7777 total_count: !!float 20 schema: $ref: '#/definitions/getExtractTableDataSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection proextract message: 'Forbidden: conn_name' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Table not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'extract table not found: customers' messageID: OPTTC0208E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: get extract Table Data tags: - Extract /v1/file/collections: delete: description: | Delete a list of collection using IDs provided operationId: deleteCollections parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: connection profile in: header name: X-Connection-Profile type: string - description: Collection ID in: body name: body required: true schema: $ref: '#/definitions/deleteResourceArray' produces: - application/json responses: "200": description: OK examples: application/json: _statusCode_: !!float 200 message: Collections deleted successfully messageID: OPTTC0818I schema: $ref: '#/definitions/successResponse' "207": description: Multi status examples: application/json: _statusCode_: !!float 207 errors: - id: ce85b0d8-4139-4a1e-9728-e494e4408249 message: Collection deleted successfully - id: invalid-id-123 message: Collection not found exception: partial success message: Some collections could not be deleted messageID: OPTTC0819W schema: $ref: '#/definitions/errorResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid request message: Collection IDs are required messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: missing Bearer token message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin message: 'Forbidden: Insufficient permissions' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: Collection not found messageID: OPTTC0004E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Delete collections using IDs tags: - Collection get: description: | List all collections operationId: listCollections parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string produces: - application/json responses: "200": description: List collection successful examples: application/json: _statusCode_: !!float 200 collections: - created_at: "2024-01-15T10:30:00Z" description: Collection of customer archive files id: f34b220c-f3e8-4c48-91b3-615800bfa3e6 name: customer-data-collection updated_at: "2024-01-15T10:30:00Z" - created_at: "2024-02-20T14:45:00Z" description: Sales data archives for 2024 id: a12b340d-5678-4e90-b1c2-345678901234 name: sales-archive-2024 updated_at: "2024-02-20T14:45:00Z" message: Collections retrieved successfully messageID: OPTTC0814I schema: $ref: '#/definitions/getAllCollectionsSuccessResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'invalid UUID length: 35' message: 'Core received a bad request: Invalid collection ID' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: missing Bearer token message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin message: 'Forbidden: Insufficient permissions' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: List collections tags: - Collection post: consumes: - application/json description: | Create a group of related archive files with a descriptive name and optional description. operationId: createCollection parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: connection profile in: header name: X-Connection-Profile type: string - description: Collection payload in: body name: body required: true schema: $ref: '#/definitions/collectionPayload' produces: - application/json responses: "201": description: Create collection successful examples: application/json: _statusCode_: !!float 201 id: f34b220c-f3e8-4c48-91b3-615800bfa3e6 message: Collection 'collection-test-1' created successfully messageID: OPTTC0815I schema: $ref: '#/definitions/successPostResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid request message: Collection name is required messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: missing Bearer token message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin message: 'Forbidden: Insufficient permissions' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Failed to create collection: record not found' messageID: OPTTC0816E schema: $ref: '#/definitions/errorResponse' "409": description: Conflict examples: application/json: _statusCode_: !!float 409 errors: [] exception: record already exists message: Collection 'collection-test-3' already exists messageID: OPTTC0817E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Create a collection of archive tags: - Collection /v1/file/collections/schemas: get: description: | Retrieves schemas from collection based on the provided collection ID. operationId: listCollectionSchemas parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: connection profile in: header name: X-Connection-Profile type: string - description: Collection ID in: query name: id required: true type: string x-example: 1b05d025-9682-4dc5-88d6-0088d5c294d5 - default: !!float 0 description: Number of items to skip before starting to return results (for pagination). in: query name: offset type: integer - default: !!float 25 description: Maximum number of schemas to return in the response. in: query name: limit type: integer - description: Sort order for schema names. Use 'true' for ascending and 'false' for descending. enum: - "true" - "false" in: query name: ascending type: string produces: - application/json responses: "200": description: Collection schemas examples: application/json: _statusCode_: !!float 200 message: Schemas retrieved successfully messageID: OPTTC0821I schemas: - name: public table_count: !!float 15 - name: sales table_count: !!float 8 - name: inventory table_count: !!float 12 total_count: !!float 3 schema: $ref: '#/definitions/getSchemasSuccessResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'invalid UUID length: 35' message: 'Core received a bad request: Invalid collection ID' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: missing Bearer token message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin message: 'Forbidden: Insufficient permissions' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: Collection 'collection-01' not found messageID: OPTTC0004E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: List schemas in a collection tags: - Collection /v1/file/collections/schemas/{schema_name}/tables: get: description: | Retrieves all tables in the specified schema of the collection. operationId: listCollectionSchemaTables parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: connection profile in: header name: X-Connection-Profile type: string - description: Schema name in: path name: schema_name required: true type: string - description: Collection ID in: query name: id required: true type: string x-example: 1b05d025-9682-4dc5-88d6-0088d5c294d5 - default: !!float 0 description: The number of items to skip before starting to collect the result set in: query name: offset type: integer - default: !!float 25 description: The number of items to return in: query name: limit type: integer - description: Sort table names in ascending or descending order enum: - "true" - "false" in: query name: ascending type: string produces: - application/json responses: "200": description: List of tables examples: application/json: _statusCode_: !!float 200 message: Tables retrieved successfully messageID: OPTTC0822I tables: - name: customers row_count: !!float 1500 size_bytes: !!float 524288 - name: orders row_count: !!float 3200 size_bytes: !!float 1048576 - name: products row_count: !!float 850 size_bytes: !!float 262144 total_count: !!float 3 schema: $ref: '#/definitions/getSchemaTablesSuccessResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'invalid UUID length: 35' message: 'Core received a bad request: Invalid collection ID' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: missing Bearer token message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin message: 'Forbidden: Insufficient permissions' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: Collection 'collection-01' not found messageID: OPTTC0004E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Lists tables in given schemas of a collection tags: - Collection /v1/file/collections/schemas/{schema_name}/tables/{table_name}: get: description: "Retrieves detailed information about a specific table within given schemas of a collection. \nThe table is identified by its name in the path parameter.\n" operationId: getCollectionSchemaTable parameters: - description: Collection ID in: query name: id required: true type: string x-example: 1b05d025-9682-4dc5-88d6-0088d5c294d5 - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: connection profile in: header name: X-Connection-Profile type: string - description: Schema name in: path name: schema_name required: true type: string - description: Table name in: path name: table_name required: true type: string produces: - application/json responses: "200": description: Table information examples: application/json: _statusCode_: !!float 200 message: Table information retrieved successfully messageID: OPTTC0823I table: columns: - data_type: INTEGER is_nullable: false is_primary_key: true name: customer_id - data_type: VARCHAR is_nullable: false is_primary_key: false name: customer_name - data_type: VARCHAR is_nullable: true is_primary_key: false name: email name: customers row_count: !!float 1500 schema_name: public size_bytes: !!float 524288 schema: $ref: '#/definitions/getSchemaTableByNameSuccessResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'invalid UUID length: 35' message: 'Core received a bad request: Invalid collection ID' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: missing Bearer token message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin message: 'Forbidden: Insufficient permissions' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Table not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: Collection 'collection-01' not found messageID: OPTTC0004E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload schema: $ref: '#/definitions/errorResponse' summary: Returns information about a table in given schemas of a collection tags: - Collection /v1/file/collections/schemas/{schema_name}/tables/{table_name}/data: get: description: | Fetches data from a specific table of given schema in a collection operationId: getCollectionTableData parameters: - description: Connection ID in: query name: id required: true type: string x-example: 1b05d025-9682-4dc5-88d6-0088d5c294d5 - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: | Optional JSON string containing a `subset_parameters` array used for filtering rows. Supported operators for `structured_conditions`: - EQUALS, GREATER_THAN, LESS_THAN, GREATER_THAN_OR_EQUALS, LESS_THAN_OR_EQUALS - BETWEEN, NOT_BETWEEN, IN, NOT_IN, LIKE, NOT_LIKE Condition types: - AND, OR, "" (empty string for last condition) in: header name: X-Archive-Subset-Parameters type: string x-example: '{"subset_parameters":[{"type":"where_clause","structured_conditions":{"condition":"AND","clauses":[{"schema_name":"public","table_name":"customers","column_name":"customer_id","operator":"EQUALS","value":100}]}}]}' - description: connection profile in: header name: X-Connection-Profile type: string - description: Schema name in: path name: schema_name required: true type: string - description: Table name in: path name: table_name required: true type: string - default: !!float 0 description: The number of rows to skip before starting to collect the result set in: query name: offset type: integer - default: !!float 100 description: The number of rows to return in: query name: limit type: integer - collectionFormat: multi description: Column names to order table data. Can be repeated for multi-column sorting. in: query items: type: string name: orderByColumn type: array - collectionFormat: multi description: Sort direction for each column. Must match orderByColumn positionally. in: query items: type: boolean name: ascending type: array produces: - application/json responses: "200": description: Returned result was successful examples: application/json: _statusCode_: !!float 200 data: - created_at: "2024-01-10T08:30:00Z" customer_id: !!float 1 customer_name: John Doe email: john.doe@example.com - created_at: "2024-01-12T10:15:00Z" customer_id: !!float 2 customer_name: Jane Smith email: jane.smith@example.com - created_at: "2024-01-15T14:20:00Z" customer_id: !!float 3 customer_name: Bob Johnson email: bob.johnson@example.com limit: !!float 100 message: Table data retrieved successfully messageID: OPTTC0824I offset: !!float 0 total_count: !!float 1500 schema: $ref: '#/definitions/getTableDataSuccessResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'invalid UUID length: 35' message: 'Core received a bad request: Invalid collection ID' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: missing Bearer token message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin message: 'Forbidden: Insufficient permissions' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Table not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: Table 'customers' not found in schema 'public' messageID: OPTTC0004E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get table data of a schema from collection tags: - Collection /v1/file/collections/{id}: get: description: | Get collection details by collection id. operationId: getCollection parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Collection ID in: path name: id required: true type: string x-example: 1b05d025-9682-4dc5-88d6-0088d5c294d5 produces: - application/json responses: "200": description: Get collection successful examples: application/json: _statusCode_: !!float 200 collection: created_at: "2024-01-15T10:30:00Z" description: Collection of customer archive files id: f34b220c-f3e8-4c48-91b3-615800bfa3e6 name: customer-data-collection updated_at: "2024-01-15T10:30:00Z" message: Collection retrieved successfully messageID: OPTTC0816I schema: $ref: '#/definitions/getCollectionSuccessResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'invalid UUID length: 35' message: 'Core received a bad request: Invalid collection ID' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: missing Bearer token message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin message: 'Forbidden: Insufficient permissions' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: Collection 'collection-01' not found messageID: OPTTC0004E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get collection details tags: - Collection put: description: | Update collection details operationId: updateCollection parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: connection profile in: header name: X-Connection-Profile type: string - description: Collection ID in: path name: id required: true type: string x-example: 1b05d025-9682-4dc5-88d6-0088d5c294d5 - description: Collection details in: body name: body required: true schema: $ref: '#/definitions/collectionPayload' produces: - application/json responses: "200": description: Update collection successful examples: application/json: _statusCode_: !!float 200 collection: created_at: "2024-01-15T10:30:00Z" description: Updated collection of customer archive files id: f34b220c-f3e8-4c48-91b3-615800bfa3e6 name: customer-data-collection-updated updated_at: "2024-03-20T15:00:00Z" message: Collection updated successfully messageID: OPTTC0820I schema: $ref: '#/definitions/getCollectionSuccessResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'invalid UUID length: 35' message: 'Core received a bad request: Invalid collection ID' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: missing Bearer token message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin message: 'Forbidden: Insufficient permissions' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: Collection 'collection-01' not found messageID: OPTTC0004E schema: $ref: '#/definitions/errorResponse' "409": description: Conflict examples: application/json: _statusCode_: !!float 409 errors: [] exception: record already exists message: Collection 'collection-01' already exists messageID: OPTTC0817E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: internal server error message: An unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Update collection details tags: - Collection /v1/file/job: delete: consumes: - application/json description: | Delete a Archive File Job job or list of jobs by ID. operationId: deleteArchiveFileJob parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Job IDs to be delete in: body name: body required: true schema: $ref: '#/definitions/deleteResourceArray' produces: - application/json responses: "200": description: Job deleted examples: application/json: _statusCode_: !!float 200 message: 'Successfully deleted archive file job(s): 0dfb8b8b-fc93-42d1-b193-3d7d8c92b861' messageID: OPTTC0804I schema: $ref: '#/definitions/successResponse' "207": description: Multiple Status examples: application/json: _statusCode_: !!float 207 errors: - id: 0dfb8b8b-fc93-3d7d8c92b86c reason: record not found status: failed_to_delete - id: 0dfb8b8b-fc93-3d7d8c92b863 reason: record not found status: failed_to_delete exception: 'partial_delete_error: one or more job IDs could not be deleted due to missing or invalid references' message: 'Failed to delete archive file job(s), Error: Some jobs could not be deleted' messageID: OPTTC0804E schema: $ref: '#/definitions/errorResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Delete Archive File Job tags: - Archive get: description: Get archive file by jobID operationId: getArchiveFileJob parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: The id of the archive file to retrieve. in: query name: id required: true type: string produces: - application/json responses: "200": description: Archive files retrieved successfully examples: application/json: _statusCode_: !!float 200 message: 'Archive file job retrieved successfully, jobID: 0dfb8b8b-fc93-42d1-b193-3d7d8c92b861' messageID: OPTTC0802I schema: $ref: '#/definitions/fileJobSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch file message: 'Forbidden: my-postgres-db' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get archive file by jobID tags: - Archive /v1/file/savedlist: get: description: List saved archive flows operationId: listArchiveFlows parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string produces: - application/json responses: "200": description: Archive files retrieved successfully examples: application/json: _statusCode_: !!float 200 message: Successfully listed saved archives messageID: OPTTC0811I requestObj: resources: - access_definition_name: access_node created_at: "2025-11-17T14:53:20.582834Z" created_by: 49ef049d-7a8d-4edf-b41a-4c98b35acce0 created_by_name: User1 Test flow_id: ff11a94f-2a8b-4b88-83d4-2c35ef0222b7 flow_name: archive_flow_001 flow_revision: "2" id: 054750db-8dac-4218-a06d-742b84ed46e2 job_id: 9186134c-8431-4e4a-88b2-1f3555d9a032 name: archive_001 status: FINISHED submission_id: driver-20251117145320-0000 total_count: !!float 1 schema: $ref: '#/definitions/listArchiveFlowsSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch file message: 'Forbidden: my-postgres-db' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: List saved archive flows tags: - Archive /v1/file/schemas: get: description: Get archive schemas from flow operationId: getArchiveSchemas parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: The id of the flow to retrieve. in: query name: id required: true type: string produces: - application/json responses: "200": description: Archive files retrieved successfully examples: application/json: _statusCode_: !!float 200 message: Successfully retrieved archive schemas messageID: OPTTC0805I requestObj: schemas: - name: public tables: - name: customers - name: email_address - name: orders schema: $ref: '#/definitions/getArchiveSchemasResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch file message: 'Forbidden: my-postgres-db' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Archive flow not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Flow details not found, flowID: 0dfb8b8b-d7d8c-92b861' messageID: OPTTC0322E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get archive schemas tags: - Archive /v1/file/schemas/{schema_name}/tables/{table_name}: get: description: View details about a table in the specified archive operationId: getArchiveTableDetails parameters: - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string - description: Schema name in: path name: schema_name required: true type: string x-example: dbo - description: Table name in: path name: table_name required: true type: string x-example: customers - description: Job ID associated with the file schema in: query name: id required: true type: string - default: !!float 0 description: The number of items to skip before starting to collect the result set in: query name: offset type: integer - default: !!float 100 description: The number of items to return in: query name: limit type: integer produces: - application/json responses: "200": description: Table information examples: application/json: _statusCode_: !!float 200 message: 'Getting archive table details for job:7cd71244-aac0-4abc-91f6-929d7c19fa18, schema: public, table: customers' messageID: OPTTC0801I requestObj: columns: - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: first_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: last_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: email nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 100 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: phone nullable: yes type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 20 type_scale: !!float 0 type_string_codeunit: "" comments: PostgreSQL is an open source and customizable object-relational database. For the PostgreSQL connector on IBM Cloud, select IBM Cloud Databases for PostgreSQL. conn_profile_name: postgres_db name: customers pk_info: columns: - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" comments: Primary key constraint conn_profile_name: postgres_db name: customers_pkey schema_name: public table_name: customers schema: public stats: last_collected: "2025-11-20T17:47:43.496Z" rows: !!float 0 schema: $ref: '#/definitions/getArchiveTableDetailSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: conn_name' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Table not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'File table not found: customers' messageID: OPTTC0208E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: get Archive Table Details tags: - Archive /v1/file/schemas/{schema_name}/tables/{table_name}/data: get: description: View rows from a table in the specified archive operationId: getArchiveTableData parameters: - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string - description: | Optional JSON string containing a `subset_parameters` array used for filtering rows. Supported operators for `structured_conditions`: - EQUALS, GREATER_THAN, LESS_THAN, GREATER_THAN_OR_EQUALS, LESS_THAN_OR_EQUALS - BETWEEN, NOT_BETWEEN, IN, NOT_IN, LIKE, NOT_LIKE Condition types: - AND, OR, "" (empty string for last condition) in: header name: X-Archive-Subset-Parameters type: string x-example: '{"subset_parameters":[{"type":"where_clause","structured_conditions":{"condition":"AND","clauses":[{"schema_name":"public","table_name":"customers","column_name":"customer_id","operator":"EQUALS","value":100}]}}]}' - description: Schema name in: path name: schema_name required: true type: string x-example: dbo - description: Table name in: path name: table_name required: true type: string x-example: customers - description: Job ID associated with the file schema in: query name: id required: true type: string x-example: 7cd71244-aac0-4abc-91f6-929d7c19fa18 - default: !!float 0 description: The number of items to skip before starting to collect the result set in: query name: offset type: integer - default: !!float 100 description: The number of items to return in: query name: limit type: integer produces: - application/json responses: "200": description: Table information examples: application/json: _statusCode_: !!float 200 message: 'Getting archive table data for job:7cd71244-aac0-4abc-91f6-929d7c19fa18, schema: dbo, table: customers' messageID: OPTTC0803I requestObj: columns: - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: first_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: last_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: email nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 100 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: phone nullable: yes type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 20 type_scale: !!float 0 type_string_codeunit: "" rows: - - "1" - Alice - Johnson - alice@example.com - 123-456-7890 - - "2" - Bob - Smith - bob@example.com - 987-654-3210 - - "3" - Charlie - Davis - charlie@example.com - 555-666-7777 total_count: !!float 20 schema: $ref: '#/definitions/getArchiveTableDataSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: User is neither the owner nor an admin; cannot fetch connection profile message: 'Forbidden: conn_name' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Table not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'File table not found: customers' messageID: OPTTC0208E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: get Archive Table Data tags: - Archive /v1/flow: delete: consumes: - application/json description: | Delete flows using the IDs operationId: deleteFlow parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Flow IDs to be delete in: body name: body required: true schema: $ref: '#/definitions/deleteResourceArray' produces: - application/json responses: "200": description: Flow deleted examples: application/json: _statusCode_: !!float 200 message: 'Successfully deleted flow(s): 0dfb8b8b-d7d8c-92b861' messageID: OPTTC0806I schema: $ref: '#/definitions/successResponse' "207": description: Multiple Status examples: application/json: _statusCode_: !!float 207 errors: - id: 0dfb8b8b-fc93-3d7d8c92b86c reason: record not found status: failed_to_delete - id: 0dfb8b8b-fc93-3d7d8c92b863 reason: record not found status: failed_to_delete exception: 'partial_delete_error: one or more job IDs could not be deleted due to missing or invalid references' message: 'Failed to delete flow(s), Error: Some flows could not be deleted' messageID: OPTTC0807E schema: $ref: '#/definitions/errorResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Delete flows tags: - Flow post: consumes: - application/vnd+ibm.dataplatform-pipeline+json description: | Creating a new flow using the `pipeline-flow-v3` object in JSON format, leveraging existing nodes and connection profiles in your system. ### Prerequisites Nodes must already have been created via the `/v1/node` endpoints. Connection profiles for source/target databases must already exist in the system. ### When to Use This Endpoint - When reusing existing nodes. --- ## Step 1: Create the Pipeline Skeleton Begin by establishing the foundational structure of your flow. This skeleton serves as the container for all processing logic. ```json { "id": "", "version": "3.0", "app_data": { "optim": { "version": "1.1.0" } }, "doc_type": "pipeline", "parameters": {}, "json_schema": "https://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json", "primary_pipeline": "", "pipelines": [ { "id": "", "name": "", "nodes": [ // Nodes will be added here ], "app_data": { "ui_data": { "comments": [], "flow_type": "" } }, "runtime_ref": "optim-runtime:1.1.0" } ] } ``` **Important Configuration Notes:** - Generate unique UUIDs for both the `id` and pipeline `id` fields. - Ensure the `primary_pipeline` matches the pipeline you intend to execute. - The `runtime_ref` specifies the execution environment version. --- ## Step 2: Define Connection Nodes Add connection nodes to reference source and target databases. These nodes link your pipeline to pre-configured connection profiles. ```json { "op": "connection_profile", "id": "", "type": "binding", "parameters": { "connection_type": "" }, "connection": { "ref": "", "name": "", "properties": {} } } ``` **Connection Configuration Requirements:** - Connection profiles must already exist in your environment. - Ensure you have the UUID references for each connection. - Specify whether each connection is a `source` or `target`. --- ## Step 3: Define Execution Nodes Execution nodes perform the actual data operations in the pipeline. Each node is populated using existing node data from the system. ```json { "op": "", "type": "execution_node", "id": "", "app_data": {} // Populated from existing node in system } ``` ## Finalizing the Pipeline Once all nodes are defined and added to the `nodes` array, the flow is ready for creation using the flow creation endpoint. operationId: createFlow parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Flow, outer JSON of pipeline-flow-v3 object in: body name: body required: true schema: $ref: '#/definitions/flow' produces: - application/json responses: "201": description: Flow created. examples: application/json: _statusCode_: !!float 201 message: 'Flow saved to metadb, flowID: a81735c4-a97c-40f9-9c58-d23f09f8c31b, flowName: flow-fmt-01' messageID: OPTTC0321I requestObj: id: a81735c4-a97c-40f9-9c58-d23f09f8c31b name: flow-fmt-01 schema: $ref: '#/definitions/flowCreateUpdateSuccessResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] message: Source file not found. messageID: OPTTC0331E schema: $ref: '#/definitions/errorResponse' "409": description: Flow name/id already exists examples: application/json: _statusCode_: !!float 400 errors: [] exception: record already exists message: 'Flow already exists: flow-fmt-01' messageID: OPTTC0309E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Create a new flow with existing nodes tags: - Flow put: consumes: - application/json description: | Update a flow using the new pipeline JSON operationId: updateFlow parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Flow, pipeline-flow-v3 object in: body name: body required: true schema: $ref: '#/definitions/flow' produces: - application/json responses: "200": description: Flow updated examples: application/json: _statusCode_: !!float 200 message: 'Successfully updated flow, ID: a81735c4-a97c-40f9-9c58-d23f09f8c31b, Name: flow-fmt-01' messageID: OPTTC0325I requestObj: id: a81735c4-a97c-40f9-9c58-d23f09f8c31b name: flow-fmt-01 schema: $ref: '#/definitions/flowCreateUpdateSuccessResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Flow name/id not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Flow not found: Name: flow_01, Error: Error message' messageID: OPTTC0326E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Update flow tags: - Flow /v1/flow/create-nodes: post: consumes: - application/vnd+ibm.dataplatform-pipeline+json description: | Creates a new flow using the `pipeline-flow-v3` JSON schema, leveraging existing connection profiles and creating new nodes as needed in your system. If an existing node is referenced in the pipeline, the system will use the existing node stored in the system. The node is referenced by name and id. This endpoint will not edit/update an existing node. ### Prerequisites Connection profiles for source/target databases must already exist in the system. ### Note - All execution nodes (access_definition, subset_definition, transform_definition, relationship_definition, insert_definition) will be saved to the metadb. - `app_data` of the nodes **MUST be complete** with all required configuration. - For more details regarding nodes, refer to POST `/v1/node/{node_type}` ### When to Use This Endpoint - To create a complete flow from scratch with new and/or existing node definitions. --- ## Step 1: Create the Pipeline Skeleton Begin by establishing the foundational structure of your flow. This skeleton serves as the container for all processing logic. ```json { "id": "", "version": "3.0", "app_data": { "optim": { "version": "1.1.0" } }, "doc_type": "pipeline", "parameters": {}, "json_schema": "https://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json", "primary_pipeline": "", "pipelines": [ { "id": "", "name": "", "nodes": [ // Nodes will be added here ], "app_data": { "ui_data": { "comments": [], "flow_type": "" } }, "runtime_ref": "optim-runtime:1.1.0" } ] } ``` **Important Configuration Notes:** - Generate unique UUIDs for both the `id` and pipeline `id` fields. - Ensure the `primary_pipeline` id matches the pipeline you intend to execute (only one pipeline may be included in a flow at this time). - The `runtime_ref` specifies the execution environment version. --- ## Step 2: Add Connection Nodes Add connection nodes to reference source and target databases. These nodes link your pipeline to pre-configured connection profiles. The properties object is populated when the flow sumbitted for execution. ```json { "op": "connection_profile", "id": "", "type": "binding", "parameters": { "connection_type": "" }, "connection": { "ref": "", "name": "", "properties": {} } } ``` **Connection Configuration Requirements:** - Connection profiles must already exist in your environment. - Ensure you have the UUID references for each connection. - Specify whether each connection is a `source` or `target`. --- ## Step 3: Define Execution Nodes Execution nodes perform the actual data operations in the pipeline. Each node is to be populated with data to be processed. Refer nodes section for more details. ```json { "op": "", "type": "execution_node", "id": "", "app_data": { // Node data to be included here. } } ``` ## Finalizing the Pipeline Once all nodes are defined and added to the `nodes` array, the flow is ready for creation using the flow creation endpoint. operationId: createFlowWithNodes parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Flow, outer JSON of pipeline-flow-v3 object in: body name: body required: true schema: $ref: '#/definitions/flow' produces: - application/json responses: "201": description: Flow created. examples: application/json: _statusCode_: !!float 201 message: 'Flow saved to metadb, flowID: b92846d5-b08d-51g0-0d69-e34g10g9d42c, flowName: flow-with-nodes-01' messageID: OPTTC0321I requestObj: id: b92846d5-b08d-51g0-0d69-e34g10g9d42c name: flow-with-nodes-01 schema: $ref: '#/definitions/flowCreateUpdateSuccessResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] message: Source file not found. messageID: OPTTC0331E schema: $ref: '#/definitions/errorResponse' "409": description: Flow name/id already exists examples: application/json: _statusCode_: !!float 400 errors: [] exception: record already exists message: 'Flow already exists: flow-fmt-01' messageID: OPTTC0309E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Create a new flow and create each included node. tags: - Flow /v1/flow/savedlist: get: description: | List all saved flows. Administrators can retrieve the list of all flows in the system. Regular users will receive a list flows they have created. operationId: listSavedFlows parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Type name to match in: query name: worksheet_type type: string - description: Source connenction name to match in: query name: src_conn_name type: string produces: - application/json responses: "200": description: List of saved flows examples: application/json: _statusCode_: !!float 200 flows: - id: a81735c4-a97c-40f9-9c58-d23f09f8c31b name: flow-fmt-01 worksheet_type: archive - id: b92846d5-b08d-51g0-0d69-e34g10g9d42c name: flow-with-nodes-01 worksheet_type: masking message: Successfully listed saved flows messageID: OPTTC0324I schema: $ref: '#/definitions/listSavedFlowsSuccessResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: List saved flows tags: - Flow /v1/flow/submit/{id}: post: description: "Submits the specified flow by ID to the runtime environment for execution. Ensure the flow has been created and saved before submission. \nOnce submitted, the flow will be processed according to its defined pipelines and logic. Also, a copy of the submitted flow is preserved in the metadata.\n" operationId: submitFlow parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: UUID of the flow to submit for execution. in: path name: id required: true type: string produces: - application/json responses: "202": description: Flow accepted by Spark examples: application/json: _statusCode_: !!float 202 jobID: c03957e6-c19e-52h1-1e70-f45h21h0e53d message: Job submitted successfully. messageID: OPTTC0308I submissionID: driver-20251120182530-0001 schema: $ref: '#/definitions/flowSubmitSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred schema: $ref: '#/definitions/errorResponse' summary: Submit a flow for execution tags: - Flow /v1/flow/{id}: get: description: | Get a flow by ID. Administrators can retrieve the any flows in the system. Regular users will be able to retrieve flow they have created. operationId: getSavedFlow parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: The id of the flow to retrieve. in: path name: id required: true type: string produces: - application/json responses: "200": description: Flow retrieved successfully examples: application/json: _statusCode_: !!float 200 message: 'Flow details retrieved successfully, flowID: a81735c4-a97c-40f9-9c58-d23f09f8c31b, revision: 1' messageID: OPTTC0323I requestObj: id: a81735c4-a97c-40f9-9c58-d23f09f8c31b name: flow-fmt-01 revision: !!float 1 schema: $ref: '#/definitions/flowDetailsSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: The job does not exist examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Flow details not found, flowID: a9882e61-34c9-485e-9eaa-1bda46a2d0df' messageID: OPTTC0321E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get a flow by ID tags: - Flow /v1/job: delete: consumes: - application/json description: | Delete a job or list of jobs by ID. Administrators can delete any job in the system. Regular users can only delete jobs they own or have been granted access to. operationId: deleteJobs parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Job IDs to be delete in: body name: body required: true schema: $ref: '#/definitions/deleteResourceArray' produces: - application/json responses: "200": description: Job deleted examples: application/json: _statusCode_: !!float 200 message: 'Successfully deleted job(s): 0dfb8b8b-d7d8c-92b861' messageID: OPTTC0303I schema: $ref: '#/definitions/successResponse' "207": description: Multiple Status examples: application/json: _statusCode_: !!float 207 errors: - id: 0dfb8b8b-fc93-3d7d8c92b86c reason: record not found status: failed_to_delete - id: 0dfb8b8b-fc93-3d7d8c92b863 reason: record not found status: failed_to_delete exception: 'partial_delete_error: one or more job IDs could not be deleted due to missing or invalid references' message: 'Failed to delete job(s), Error: Some jobs could not be deleted' messageID: OPTTC0303E schema: $ref: '#/definitions/errorResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Delete job tags: - Job /v1/job/details/{id}: get: description: | Get the details of a specific job. operationId: getSavedJobDetails parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: The id of the job to retrieve. in: path name: id required: true type: string produces: - application/json responses: "200": description: Job details retrieved successfully examples: application/json: _statusCode_: !!float 200 job: created_at: "2024-01-15T10:30:00Z" created_by: user@example.com dest_conn_id: f1e2d3c4-b5a6-7890-cdef-ab9876543210 dest_conn_name: s3-archive id: 0dfb8b8b-fc93-42d1-b193-3d7d8c92b861 name: customer-data-extract pipeline_config: tables: - row_count: !!float 1500 schema: public table: customers src_conn_id: a1b2c3d4-e5f6-7890-abcd-ef1234567890 src_conn_name: postgres-prod status: completed updated_at: "2024-01-15T10:30:00Z" worksheet_type: extract message: Job details retrieved successfully messageID: OPTTC0304I schema: $ref: '#/definitions/jobDetailsSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: The job does not exist examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Job details not found, jobID: a9882e61-34c9-485e-9eaa-1bda46a2d0df' messageID: OPTTC0302E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get saved job details tags: - Job /v1/job/preview/{id}: get: description: | Get a preview of rows on subsetting and masking inputs. operationId: getRowPreview parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: The id of the job to retrieve. in: path name: id required: true type: string produces: - application/json responses: "200": description: Sample rows retrieved successfully examples: application/json: _statusCode_: !!float 200 message: Preview rows retrieved successfully with masking applied messageID: OPTTC0305I preview_data: - customer_id: !!float 1 customer_name: John Doe email: j***@example.com phone: '***-***-1234' - customer_id: !!float 2 customer_name: Jane Smith email: j***@example.com phone: '***-***-5678' - customer_id: !!float 3 customer_name: Bob Johnson email: b***@example.com phone: '***-***-9012' total_rows: !!float 3 schema: $ref: '#/definitions/previewRowsSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Display sample rows tags: - Job /v1/job/report/spark_app_id/{spark_app_id}/flow_id/{flow_id}: get: description: | Retrieve process report from optim-runtime using Spark application ID and flow ID. This endpoint forwards the request to the runtime service. User must be the flow creator or an administrator to access the report. operationId: getProcessReport parameters: - description: Account ID in: header name: account-id type: string - description: Access credentials in the form of a JWT token in: header name: Authorization required: true type: string - description: Spark application ID (e.g., app-20260303053047-0000) in: path name: spark_app_id required: true type: string - description: Flow ID (UUID format) in: path name: flow_id required: true type: string produces: - application/json responses: "200": description: Successfully retrieved process report examples: application/json: _statusCode_: !!float 200 flow_id: f34b220c-f3e8-4c48-91b3-615800bfa3e6 message: Process report retrieved successfully messageID: OPTTC0309I report: execution_time_seconds: !!float 885 status: completed tables_processed: - records_filtered: !!float 500 records_masked: !!float 3000 records_processed: !!float 5000 schema_name: public table_name: customers - records_filtered: !!float 700 records_masked: !!float 5500 records_processed: !!float 10000 schema_name: public table_name: orders total_records_filtered: !!float 1200 total_records_masked: !!float 8500 total_records_processed: !!float 15000 spark_app_id: app-20240315103047-0000 schema: $ref: '#/definitions/processReportSuccessResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid parameters message: 'Core received a bad request: Invalid Spark App ID or Flow ID format' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Insufficient permissions' message: Access is forbidden for user. Only flow creator or admin can access this report. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Report Not Found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Process report not found for Spark App ID: app-20240315103047-0000' messageID: OPTTC0004E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred while retrieving process report messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get process report by Spark App ID and Flow ID tags: - Job /v1/job/savedlist: get: description: | List all saved jobs. Administrators can retrieve the list of all jobs in the system. Regular users will receive a list to which they have accessibility operationId: listSavedJobs parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Type name to match in: query name: worksheet_type type: string - description: Source connenction name to match in: query name: src_conn_name type: string produces: - application/json responses: "200": description: List of saved jobs examples: application/json: _statusCode_: !!float 200 jobs: - created_at: "2024-01-15T10:30:00Z" id: 0dfb8b8b-fc93-42d1-b193-3d7d8c92b861 name: customer-data-extract src_conn_name: postgres-prod status: completed updated_at: "2024-01-15T10:30:00Z" worksheet_type: extract - created_at: "2024-02-20T14:45:00Z" id: b7cd199d-fe8f-4a63-9333-497f022f20ee name: sales-archive-job src_conn_name: mysql-sales status: running updated_at: "2024-02-20T14:45:00Z" worksheet_type: archive message: Jobs retrieved successfully messageID: OPTTC0301I schema: $ref: '#/definitions/jobListSuccessResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: List saved jobs tags: - Job /v1/job/spark/status/{id}: get: description: | Retreive and update run statuses for the provided job ID. operationId: getSparkRunStatus parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - default: !!float 25 description: Number of prior runs to retrieve and update in: query name: limit type: integer - description: The job id to get run statuses for. in: path name: id required: true type: string produces: - application/json responses: "200": description: List of Spark run statuses examples: application/json: _statusCode_: !!float 200 message: Spark run statuses retrieved successfully messageID: OPTTC0306I runs: - completed_at: "2024-03-15T10:45:30Z" duration_seconds: !!float 885 spark_app_id: app-20240315103047-0000 started_at: "2024-03-15T10:30:45Z" status: FINISHED submission_id: driver-20240315103045-0001 - completed_at: "2024-03-14T15:35:10Z" duration_seconds: !!float 290 error_message: Connection timeout spark_app_id: app-20240314153022-0001 started_at: "2024-03-14T15:30:20Z" status: FAILED submission_id: driver-20240314153020-0002 - spark_app_id: app-20240313092017-0002 started_at: "2024-03-13T09:20:15Z" status: RUNNING submission_id: driver-20240313092015-0003 schema: $ref: '#/definitions/sparkRunStatusSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: The job does not exist examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: 'Job details not found, jobID: a9882e61-34c9-485e-9eaa-1bda46a2d0df' messageID: OPTTC0302E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get run statuses for a job tags: - Job /v1/job/spark/submission/kill/{id}: post: description: Spark submission can be killed using driver ID operationId: killSubmission parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Spark submission ID in: path name: id required: true type: string produces: - application/json responses: "200": description: Kill request sent. examples: application/json: _statusCode_: !!float 200 message: Spark submission kill request sent successfully messageID: OPTTC0308I status: KILLED submission_id: driver-20240315103045-0001 schema: $ref: '#/definitions/killSubmissionSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Kill a Spark submission tags: - Job /v1/job/spark/{id}: get: description: | Retreive spark status for a given submission ID. operationId: getSparkStatus parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Job submission id in: path name: id required: true type: string produces: - application/json responses: "200": description: Successfully fetched run examples: application/json: _statusCode_: !!float 200 driver_state: RUNNING message: Spark run status retrieved successfully messageID: OPTTC0307I spark_app_id: app-20240315103047-0000 started_at: "2024-03-15T10:30:45Z" status: RUNNING submission_id: driver-20240315103045-0001 worker_state: ALIVE schema: $ref: '#/definitions/getSparkRunSuccessResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid submission ID message: 'Core received a bad request: Invalid submission ID format' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get spark status tags: - Job /v1/maintenance/disk/prune: post: consumes: - application/json description: | Clean up old records from metadb tables to free up disk space. This operation can be run in dry-run mode to preview the impact before actual deletion. Available retention policies: AGE_BASED - deletes records older than a threshold (Default: 30 days) TABLE_DISK_USAGE_BASED - prunes when table storage usage exceeds a limit given in bytes (Default: 10000 bytes) operationId: postMaintenanceDiskPrune parameters: - description: Disk prune configuration in: body name: body required: true schema: $ref: '#/definitions/diskPruneRequest' produces: - application/json responses: "200": description: Disk prune operation completed successfully examples: application/json: _statusCode_: !!float 200 dry_run: false execution_time_seconds: 45.2 message: Disk prune operation completed successfully messageID: OPTTC0901I tables_pruned: - disk_space_freed_mb: 245.5 records_deleted: !!float 15420 table_name: job_runs - disk_space_freed_mb: 142.8 records_deleted: !!float 8930 table_name: flow_executions - disk_space_freed_mb: 512.3 records_deleted: !!float 45678 table_name: audit_logs total_disk_space_freed_mb: 900.6 total_records_deleted: !!float 70028 schema: $ref: '#/definitions/diskPruneSuccessResponse' "400": description: Bad request - invalid parameters examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid retention period message: 'Core received a bad request: Retention period must be a positive integer' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized - invalid or missing authentication token examples: application/json: _statusCode_: !!float 401 errors: [] exception: missing Bearer token message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden - insufficient permissions examples: application/json: _statusCode_: !!float 403 errors: [] exception: User does not have admin privileges message: 'Forbidden: Only administrators can perform maintenance operations' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal server error examples: application/json: _statusCode_: !!float 500 errors: [] exception: database connection failed message: An unexpected error occurred during disk prune operation messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Prune old records from metadb tags: - Maintenance /v1/masking/categories: get: description: Retrieve the available masking types and formats. operationId: getMaskingCategories parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization type: string - description: account id in: header name: account-id type: string responses: "200": description: Masking types and formats examples: application/json: _statusCode_: !!float 200 message: Successfully retrieved masking formats and types messageID: OPTTC0601I requestObj: masking_formats: Address: StreetSuffix: - RepeatableFormatFabricationProcessor - FormatPreservingTokenizationProcessor - FormatPreservingRedactionProcessor - RandomFormatFabricationProcessor USAddressSimple: - RandomFormatFabricationProcessor - RepeatableFormatFabricationProcessor CreditCardInformation: AmericanExpressCard: - RandomFormatFabricationProcessor - RepeatableFormatFabricationProcessor CreditCard: - FormatPreservingRedactionProcessor - RandomFormatFabricationProcessor - RepeatableFormatFabricationProcessor - FormatPreservingTokenizationProcessor pkey_types: - RepeatableFormatFabricationProcessor - FormatPreservingTokenizationProcessor - RepeatableNoiseProcessor schema: $ref: '#/definitions/maskingCategoriesSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get available masking types and formats tags: - Masking /v1/masking/format: get: description: Retrieve all masking format operationId: listMaskingFormats parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string responses: "200": description: List of masking format examples: application/json: _statusCode_: !!float 200 message: 'Retrieved masking formats successfully, count: 2' messageID: OPTTC0613I requestObj: formats: - description: 'Masking format sample ' id: d3d1174e-d606-49f0-955b-02e4de0c1f27 last_updated: "2026-01-15T12:01:02-08:00" name: Masking format sample 1 user_id: a1b2c3d4-e5f6-7890-g1h2-i3j4k5l6m7n8 user_name: user1 - description: Masking format sample 2 id: d3d1174e-d606-49f0-955b-02e4de0c1f28 last_updated: "2026-01-30T09:02:17-08:00" name: Masking format sample 2 user_id: ferqw345-3456-7890-g1h2-i3j4k5l6m7n8 user_name: user2 schema: $ref: '#/definitions/listMaskingFormatsSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Internal error message: Internal server error messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Unexpected Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: List all masking format tags: - Masking post: consumes: - application/json description: | Create a new masking format Skeleton for request body: ```json { "id": "", "name": "", "description": "" "value": { "id": "FixedLengthFormat", "format": { "type": "FixedLengthString", "configuration": { "alphabet": "c", "length": 4 } } } } ``` operationId: createMaskingFormat parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string - description: Masking format payload in: body name: body required: true schema: $ref: '#/definitions/maskingFormat' produces: - application/json responses: "201": description: Masking format created successfully examples: application/json: _statusCode_: !!float 201 message: 'Masked format saved to metadb, id: d3d1174e-d606-49f0-955b-02e4de0c1f27, name: Masking format sample' messageID: OPTTC0614I requestObj: id: d3d1174e-d606-49f0-955b-02e4de0c1f27 name: Masking format sample schema: $ref: '#/definitions/maskingFormatCreateUpdateSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "409": description: Conflict - Format name already exists examples: application/json: _statusCode_: !!float 409 errors: [] exception: 'Conflict: Format name already exists' message: Masking format with this name already exists messageID: OPTTC0615E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Create a masking format tags: - Masking put: consumes: - application/json description: Update an existing masking format operationId: updateMaskingFormat parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string - description: Masking format payload in: body name: body required: true schema: $ref: '#/definitions/maskingFormat' produces: - application/json responses: "200": description: Masking format updated successfully examples: application/json: _statusCode_: !!float 200 message: 'Successfully updated masking format, id: d3d1174e-d606-49f0-955b-02e4de0c1f27, name: Masking format sample' messageID: OPTTC0607I requestObj: id: d3d1174e-d606-49f0-955b-02e4de0c1f27 name: Masking format sample schema: $ref: '#/definitions/maskingFormatCreateUpdateSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Masking format not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: 'Not Found: Masking format not found' message: Masking format not found messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "409": description: Conflict - Masking format name already exists examples: application/json: _statusCode_: !!float 409 errors: [] exception: 'Conflict: Format name already exists' message: Masking format with this name already exists messageID: OPTTC0606E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Update a masking Format tags: - Masking /v1/masking/format/{id}: delete: description: Delete a masking format by its ID operationId: deleteMaskingFormat parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string - description: The id of the masking format to delete. in: path name: id required: true type: string responses: "200": description: Masking format deleted successfully examples: application/json: _statusCode_: !!float 200 message: Masking format deleted successfully messageID: "" schema: $ref: '#/definitions/successResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] exception: 'Not Found: Masking format not found' message: Masking format not found messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Delete a masking format tags: - Masking get: description: Retrieve a specific masking format by its ID operationId: getMaskingFormat parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string - description: The id of the masking format to retrieve. in: path name: id required: true type: string responses: "200": description: Masking format details examples: application/json: _statusCode_: !!float 200 message: 'Masking format retrieved successfully, id: d3d1174e-d606-49f0-955b-02e4de0c1f27' messageID: OPTTC0608I requestObj: format: description: Masking format sample id: d3d1174e-d606-49f0-955b-02e4de0c1f27 name: Masking format sample value: type: string last_updated: "2026-01-14T10:10:10-00:00" user_id: 49ef049d-7a8d-4edf-b41a-4c98b35acce0 user_name: user1 schema: $ref: '#/definitions/maskingFormatSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] exception: 'Not Found: Masking format not found' message: Masking format not found messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Internal error message: Internal server error messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Unexpected Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get a masking format by ID tags: - Masking /v1/masking/policy: get: description: Retrieve all masking policies operationId: listMaskingPolicies parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string responses: "200": description: List of masking policies examples: application/json: _statusCode_: !!float 200 message: 'Retrieved masking policies successfully, count: 2' messageID: OPTTC0613I requestObj: policies: - last_updated: "2023-09-27T12:00:00Z" policy: description: Masking policy sample 1 id: 550e8400-e29b-41d4-a716-446655440000 locked: false name: Masking policy sample 1 type: optim_policy user_id: 885f509-ea59-43fc-b1ad-f050 user_name: user1 - last_updated: "2023-09-27T12:00:00Z" policy: description: Masking policy sample 2 id: 550e8400-e29b-41d4-a716-446655440001 locked: false name: Masking policy sample 2 type: runtime user_id: 885f509-ea59-43fc-b1ad-f050 user_name: user1 schema: $ref: '#/definitions/listMaskingPoliciesSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Internal error message: Internal server error messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Unexpected Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: List all masking policies tags: - Masking post: consumes: - application/json description: | Create a new masking policy Skeleton for request body: ```json { "id": "", "name": "", "description": "", "locked": false, "type": "optim_policy", "value": { "doc_type": "pipeline", "version": "3.0", "json_schema": "http://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json", "id": "0b67fa0c-8ebc-4e29-ad09-f898d87a8f08", "name": "masking_policy", "primary_pipeline": "5a96dfcc-22d1-4301-9da1-0eb2072644b5", "pipelines": [ { "id": "5a96dfcc-22d1-4301-9da1-0eb2072644b5", "name": "Test", "nodes": [ { "id": "d6c60cd8-aa14-4cfb-904c-6fc25d0d5a15", "type": "execution_node", "op": "processor_definition", "app_data": { "processor": { "id": "Text", "type": "TextProcessor", "root": true, "configuration": { "preProcessorConfiguration": { "javaScript": "function process() { return payload; }" } }, "loggable": true }, "originalId": "text", "ui_data": {} }, "inputs": [], "outputs": [] } ], "app_data": { "expressions": [], "formats": [] }, "runtime_ref": "" } ], "schemas": [] } } ``` operationId: createMaskingPolicy parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string - description: Masking policy payload in: body name: body required: true schema: $ref: '#/definitions/maskingPolicy' produces: - application/json responses: "201": description: Masking policy created successfully examples: application/json: _statusCode_: !!float 201 message: 'Masked policy saved to metadb, id: 550e8400-e29b-41d4-a716-446655440000, name: Masking policy sample' messageID: OPTTC0603I requestObj: id: 550e8400-e29b-41d4-a716-446655440000 name: Masking policy sample schema: $ref: '#/definitions/maskingPolicyCreateUpdateSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "409": description: Conflict - Policy name already exists examples: application/json: _statusCode_: !!float 409 errors: [] exception: 'Conflict: Policy name already exists' message: Masking policy with this name already exists messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Create a masking policy tags: - Masking put: consumes: - application/json description: Update an existing masking policy operationId: updateMaskingPolicy parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string - description: Masking policy payload in: body name: body required: true schema: $ref: '#/definitions/maskingPolicy' produces: - application/json responses: "200": description: Masking policy updated successfully examples: application/json: _statusCode_: !!float 200 message: 'Successfully updated masking policy, id: 550e8400-e29b-41d4-a716-446655440000, name: Masking policy sample' messageID: OPTTC0607I requestObj: id: 550e8400-e29b-41d4-a716-446655440000 name: Masking policy sample schema: $ref: '#/definitions/maskingPolicyCreateUpdateSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Masking policy not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: 'Not Found: Masking policy not found' message: Masking policy not found messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "409": description: Policy is locked examples: application/json: _statusCode_: !!float 409 errors: [] exception: masking policy locked message: Cannot update locked masking policy. Locked policies are immutable. messageID: OPTTC0612E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Update a masking policy tags: - Masking /v1/masking/policy/{id}: delete: description: Delete a masking policy by its ID operationId: deleteMaskingPolicy parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string - description: The id of the masking policy to delete. in: path name: id required: true type: string responses: "200": description: Masking policy deleted successfully examples: application/json: _statusCode_: !!float 200 message: Masking policy deleted successfully messageID: "" schema: $ref: '#/definitions/successResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] exception: 'Not Found: Masking policy not found' message: Masking policy not found messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "409": description: Policy is locked examples: application/json: _statusCode_: !!float 409 errors: [] exception: masking policy locked message: Cannot delete locked masking policy. Locked policies are immutable. messageID: OPTTC0611E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error schema: $ref: '#/definitions/errorResponse' summary: Delete a masking policy tags: - Masking get: description: Retrieve a specific masking policy by its ID operationId: getMaskingPolicy parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: account id in: header name: account-id type: string - description: The id of the masking policy to retrieve. in: path name: id required: true type: string responses: "200": description: Masking policy details examples: application/json: _statusCode_: !!float 200 message: 'Masking policy retrieved successfully, id: 550e8400-e29b-41d4-a716-446655440000' messageID: OPTTC0608I requestObj: last_updated: "2026-01-08T18:21:19-08:00" policy: description: Masking policy sample description id: 550e8400-e29b-41d4-a716-446655440000 locked: false name: Masking policy sample type: optim_policy value: {} user_id: 49ef049d-7a8d-4edf-b41a-4c98b35acce0 user_name: user1 schema: $ref: '#/definitions/maskingPolicySuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Not Found examples: application/json: _statusCode_: !!float 404 errors: [] exception: 'Not Found: Masking policy not found' message: Masking policy not found messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Internal error message: Internal server error messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Unexpected Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get a masking policy by ID tags: - Masking /v1/masking/text/process: post: description: Process text operationId: getMaskedText parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization type: string - description: account id in: header name: account-id type: string - description: Payload for masking process in: body name: body required: true schema: $ref: '#/definitions/maskTextPayload' responses: "200": description: Success examples: application/json: message: Successfully masked text messageID: OPTTC0601I requestObj: - Маршалловы Острова - 澤西 statusCode: !!float 200 schema: $ref: '#/definitions/maskedTextSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Process text tags: - Masking /v1/masking/types: get: description: Retrieve the available masking types and formats. operationId: getMaskingTypes parameters: - description: Access credentials in the form of a JWT token. This is the preferred method of supplying access credentials. in: header name: Authorization type: string - description: account id in: header name: account-id type: string responses: "200": description: Masking types and formats examples: application/json: message: Successfully retrieved masking formats and types messageID: OPTTC0601I requestObj: masking_formats: Address: - StreetSuffix - StreetSuffix_CaseSensitive - USAddressSimple - USAddressWithSecondary - USStreetNames - USStreetNames_CaseSensitive - USZipCodes - USZipCodes_CaseSensitive Email: - EmailDomainSuffix_CaseSensitive - EnglishEmailCaseSensitive - NameBasedEmail pkey_types: - RepeatableFormatFabricationProcessor - FormatPreservingTokenizationProcessor - RepeatableNoiseProcessor types: - FormatPreservingRedactionProcessor - RandomFormatFabricationProcessor - RepeatableFormatFabricationProcessor - FormatPreservingTokenizationProcessor - RandomNoiseProcessor - RepeatableNoiseProcessor statusCode: !!float 200 schema: $ref: '#/definitions/maskingTypesSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: 'Core received a bad request:' message: 'Core received a bad request:' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: 'token is invalid: Token is expired' message: Invalid authentication token. Please verify your token messageID: OPTTC0503E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: 'Forbidden: Access is forbidden for user.' message: Access is forbidden for user. messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get available masking types and formats tags: - Masking /v1/node/access: post: consumes: - application/json description: | Create new access definition node. Skeleton for request body: ```json { "op": "access_definition", "type": "execution_node", "id": "", "app_data": { "source_tables": [ { "connection_profile": "", "database": "", "id": "", "schema": "SCHEMA_NAME", "table_name": "TABLE_NAME" "columns": ["COLUMN_NAME1", "COLUMN_NAME2", "COLUMN_NAME3"], "exclude_columns_on_create": "true" } ] } } ``` By default, all columns are included. To include only specific columns, specify their names in the columns field of the access definition. If a column is excluded, no data is read from or written to it. As a result: - For database targets, the underlying database populates the column with its default value (typically NULL). - For file-based targets, the column is omitted from the generated Parquet files. Note that excluded columns remain part of the schema by default, for all targets. To remove them entirely from the target schema, set the exclude_columns_on_create field to "true". This ensures the column is not created in the target schema. Example: ```json { "op": "access_definition", "type": "execution_node", "id": "9694c125-4f7f-4e3d-80a9-661df0a1c2be", "app_data": { "name": "sales-def-12", "source_tables": [ { "id": "91f32c4e-8ac1-4c2a-9ce2-11ab5b8937de", "database": "mysql", "table_name": "employee_data", "schema": "testschema_db2file_src", "connection_profile": "348c7b00-972d-4e59-90c2-c230ac8b9d41", "columns": ["emp_id", "first_name", "last_name"], "exclude_columns_on_create": "true" }, { "id": "0914cd06-30ca-419f-be21-c0a64e8e71e1", "database": "mysql", "table_name": "department_data", "schema": "testschema_db2file_src", "connection_profile": "348c7b00-972d-4e59-90c2-c230ac8b9d41" } ] } } ``` operationId: createAccessNode parameters: - description: Access credentials in the form of a jwt token. in: header name: Authorization required: true type: string - description: Node definition in: body name: body required: true schema: $ref: '#/definitions/node' example: app_data: name: sales-def-12 source_tables: - connection_profile: 348c7b00-972d-4e59-90c2-c230ac8b9d41 database: mysql id: 91f32c4e-8ac1-4c2a-9ce2-11ab5b8937de schema: retail table_name: orders - connection_profile: 348c7b00-972d-4e59-90c2-c230ac8b9d41 database: mysql id: 27abc7dd-4fd3-4ec8-85fe-4f995b3b00a8 schema: retail table_name: payments id: 9694c125-4f7f-4e3d-80a9-661df0a1c2be op: access_definition type: execution_node produces: - application/json responses: "201": description: Node created successfully examples: application/json: _statusCode_: !!float 201 id: 9694c125-4f7f-4e3d-80a9-661df0a1c2be message: Node created successfully messageID: OPTTC0701I schema: $ref: '#/definitions/successPostResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 message: 'Validation error occurred: invalid node structure' messageID: OPTTC0006E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 message: 'Forbidden: User is neither the owner nor an admin; cannot fetch node' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Access definition not found examples: application/json: _statusCode_: !!float 404 message: Record not found. Access definition does not exist messageID: OPTTC0702E schema: $ref: '#/definitions/errorResponse' "409": description: Conflict examples: application/json: _statusCode_: !!float 409 message: Node in use. Cannot create duplicate node messageID: OPTTC0703E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Create new access definition node tags: - Node /v1/node/access/{access_definition_id}/related-nodes: get: description: "Retrieve all nodes associated with a specific access definition ID.\n\nThis endpoint returns all related nodes grouped by their node type:\n- subset_definition: Subset nodes belonging to an access defintion \n- insert_definition: Insert nodes belonging to an access defintion \n- transform_definition: Transform belonging to an access defintion \n- relationship_definition: Relationship belonging to an access defintion\n" operationId: listNodes parameters: - description: Access credentials in the form of a jwt token. in: header name: Authorization required: true type: string - description: UUID of the access definition in: path name: access_definition_id required: true type: string produces: - application/json responses: "200": description: Nodes retrieved successfully schema: $ref: '#/definitions/nodeListByAccessDefinition' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 message: 'Validation error occurred: invalid access_definition_id format' messageID: OPTTC0006E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 message: 'Forbidden: User is neither the owner nor an admin; cannot fetch node' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Access definition not found examples: application/json: _statusCode_: !!float 404 message: Record not found. Access definition does not exist messageID: OPTTC0702E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: List Nodes by Access Definition ID tags: - Node /v1/node/{node_type}: post: consumes: - application/json description: "Create a new node of the specified type.\n\nThe node_type parameter determines the type of node to create:\n- subset: Creates a subset node \n- insert: Creates an insert node \n- transform: Creates a transform node \n- relationship: Creates a relationship node\n\nSkeleton payload for each node type:\n- Subset definition node: \n ```json\n {\n \"op\": \"subset_definition\",\n \"type\": \"execution_node\",\n \"id\": \"\",\n \"app_data\": {\n \"subset_tables\": [\n {\n \"schema\": \"\",\n \"database\": \"\",\n \"table_name\": \"\",\n \"source_table_id\": \"\",\n \"subset_parameters\": [\n {\n \"type\": \"where_clause\",\n \"structured_conditions\": {\n \"condition\": \"\",\n \"clauses\": [\n {\n \"value\": \"\",\n \"operator\": \"\",\n \"table_name\": \"\",\n \"column_name\": \"\",\n \"schema_name\": \"\",\n \"table_alias\": \"\"\n }\n ]\n }\n }\n ]\n }\n ]\n }\n }\n ```\n\n Example:\n ```json\n {\n \"op\": \"subset_definition\",\n \"type\": \"execution_node\",\n \"id\": \"d6149696-f6f1-434c-a6d2-7c09d584ad9b\",\n \"app_data\": {\n \"name\": \"subset_node\",\n \"subset_tables\": [\n {\n \"schema\": \"retail\",\n \"database\": \"mysql\",\n \"table_name\": \"orders\",\n \"source_table_id\": \"91f32c4e-8ac1-4c2a-9ce2-11ab5b8937de\",\n \"subset_parameters\": [\n {\n \n \"structured_conditions\": {\n \"clauses\": [\n {\n \"value\": \"'intransit'\",\n \"operator\": \"<>\",\n \"table_name\": \"orders\",\n \"column_name\": \"order_status\",\n \"schema_name\": \"retail\",\n \"table_alias\": \"\"\n },\n {\n \"value\": \"'completed'\",\n \"operator\": \"=\",\n \"table_name\": \"orders\",\n \"column_name\": \"status\",\n \"schema_name\": \"retail\",\n \"table_alias\": \"\"\n },\n ]\n \"condition\": \"and\"\n },\n \"type\": \"where_clause\"\n }\n ]\n }\n ]\n }\n }\n ```\n\n- Transform definition node: \n ```json\n {\n \"op\": \"transform_definition\",\n \"id\": \"\",\n \"type\": \"execution_node\",\n \"app_data\": {\n \"mask_tables\": [\n {\n \"schema\": \"\",\n \"database\": \"\",\n \"table_name\": \"\",\n \"source_table_id\": \"\",\n \"columns\": [\n {\n \"column_name_source\": \"\",\n \"column_name_target\": \"\",\n \"masking_properties\": {\n \"magen_function\": {\n \"name\": \"\",\n \"type\": \"\",\n \"config\": {}\n },\n \"masking_definition_id\": \"\"\n }\n }\n ]\n }\n ]\n }\n }\n ```\n\n Example:\n ```json\n {\n \"op\": \"transform_definition\",\n \"id\": \"a3b8c7d2-5e4f-4a1b-9c8d-7e6f5a4b3c2d\",\n \"type\": \"execution_node\",\n \"app_data\": {\n \"name\": \"transform_node\",\n \"mask_tables\": [\n {\n \"schema\": \"retail\",\n \"database\": \"mysql\",\n \"table_name\": \"customers\",\n \"source_table_id\": \"b4c9d8e3-6f5a-4b2c-ad9e-8f7a6b5c4d3e\",\n \"columns\": [\n {\n \"column_name_source\": \"email\",\n \"column_name_target\": \"email\",\n \"masking_properties\": {\n \"magen_function\": {\n \"name\": \"Email\",\n \"type\": \"FormatPreservingTokenizationProcessor\",\n \"config\": {}\n },\n \"masking_definition_id\": \"c5d0e9f4-7a6b-5c3d-be0f-9a8b7c6d5e4f\"\n }\n },\n {\n \"column_name_source\": \"credit_card\",\n \"column_name_target\": \"credit_card\",\n \"masking_properties\": {\n \"magen_function\": {\n \"name\": \"CreditCard\",\n \"type\": \"RedactProcessor\",\n \"config\": {}\n },\n \"masking_definition_id\": \"d6e1f0a5-8b7c-6d4e-cf1a-0b9c8d7e6f5a\"\n }\n }\n ]\n }\n ]\n }\n }\n ```\n \n **Note:** For transform_definition nodes, masking function values (`name` field and `type` field) must be retrieved from **GET /v1/masking/categories** endpoint.\n This endpoint returns available masking formats grouped by category with their supported types and subtypes.\n\n **Available Masking Methods** (one of these must be set in the `type` field):\n \n | Method | Description |\n |--------|-------------|\n | FormatPreservingRedactionProcessor | Replaces value with a constant and validates the input format |\n | RedactProcessor | Replaces value with a constant; does not validate input format |\n | RandomFormatFabricationProcessor | Generates random value in same format; not repeatable |\n | RepeatableFormatFabricationProcessor | Generates repeatable value in same format |\n | FormatPreservingTokenizationProcessor | Replaces value with unique token; repeatable and format-valid |\n | RandomNoiseProcessor | Adds noise based on domain size; non-repeatable |\n | RepeatableNoiseProcessor | Adds repeatable noise based on domain size |\n\n- Insert definition node:\n ```json\n {\n \"op\": \"insert_definition\",\n \"type\": \"execution_node\",\n \"id\": \"\",\n \"app_data\": {\n \"table_map\": [\n { \"id\": \"\"\n \"source\": {\n \"source_table_id\": \"\",\n \"connection_profile\": \"\",\n \"table_name\": \"\",\n \"schema\": \"\",\n \"database\": \"\",\n },\n \"target\": {\n \"schema\": \"\",\n \"table_name\": \"\",\n \"database\": \"\",\n \"connection_profile\": \"\"\n },\n \"column_map\": [\n {\n \"source_column\": \"\",\n \"target_column\": \"\"\n }\n ]\n }\n ],\n \"insert_parameters\": {\n \"relational_integrity\": {\n \"conflict_resolution\": \"\"\n }\n },\n \"drop_rows_post_archive\": \"true\"\n }\n }\n ```\n\n **Column Mappings:**\n \n Column mappings are defined within the insert_definition node. All mappings are specified in the column_map array. Each entry in the column_map array defines a mapping between a source and target column using the following fields:\n\n - `source_column`: Name of the source column to be mapped from.\n - `target_column`: Name of the corresponding column to be mapped to.\n\n **Drop Rows Post Archive:**\n\n To enable deletion of rows after they have been archived, set the drop_rows_post_archive field to \"true\" in the insert definition node.\n\n By default, this field is not set, and row deletion is disabled. This field has no effect on non-archive flows.\n\n Example:\n ```json\n {\n \"op\": \"insert_definition\",\n \"type\": \"execution_node\",\n \"id\": \"1440b8f4-c7a9-4512-8b51-ad47f4a2108a\",\n \"app_data\": {\n \"name\": \"insert_node\",\n \"table_map\": [\n {\n \"id\": \"7e3f4c2a-9b47-4b9c-b2e8-3c81920bd8d1\",\n \"source\": {\n \"source_table_id\": \"1b82e74d-d849-4f94-a6e2-1f79fe2a5c87\",\n \"connection_profile\": \"9c2a77f4-563e-4b07-8a37-6e31a2f30752\",\n \"table_name\": \"order_details\",\n \"schema\": \"public\",\n \"database\": \"postgres\"\n },\n \"target\": {\n \"schema\": \"public\",\n \"table_name\": \"order_details\",\n \"database\": \"postgres\",\n \"connection_profile\": \"54e7cd80-7c0b-4a47-87e0-e1b0d47fd2ad\"\n },\n \"column_map\": [\n {\n \"source_column\": \"order_id\",\n \"target_column\": \"order_id\"\n },\n {\n \"source_column\": \"customer_name\",\n \"target_column\": \"client_name\"\n }\n ]\n }\n ],\n \"insert_parameters\": {\n \"relational_integrity\": {\n \"conflict_resolution\": \"merge\"\n }\n },\n \"drop_rows_post_archive\": \"true\"\n }\n }\n ```\n\n **Conflict Resolution Options:**\n - `merge`: Insert new records, update existing.\n - `append`: Insert new records only; skip existing.\n - `update`: Update existing records only; skip new.\n\n- Relationship definition node: \n ```json\n {\n \"id\": \"\",\n \"type\": \"execution_node\",\n \"op\": \"relationship_definition\",\n \"app_data\": {\n \"relationships\": [\n {\n \"id\": \"\",\n \"parent_table_id\": \"\",\n \"child_table_id\": \"\",\n \"relationship_conditions\": [\n {\n \"parent_conditions\": {\n \"column_name\": \"\"\n },\n \"child_conditions\": {\n \"column_name\": \"\"\n }\n }\n ]\n }\n ]\n }\n }\n ```\n\n Example:\n ```json\n {\n \"id\": \"e5f2a3b4-9c8d-4e7f-ba1c-2d3e4f5a6b7c\",\n \"type\": \"execution_node\",\n \"op\": \"relationship_definition\",\n \"app_data\": {\n \"name\": \"relationship_node\",\n \"relationships\": [\n {\n \"id\": \"f6a3b4c5-0d9e-5f8a-cb2d-3e4f5a6b7c8d\",\n \"parent_table_id\": \"91f32c4e-8ac1-4c2a-9ce2-11ab5b8937de\",\n \"child_table_id\": \"27abc7dd-4fd3-4ec8-85fe-4f995b3b00a8\",\n \"relationship_conditions\": [\n {\n \"parent_conditions\": {\n \"column_name\": \"order_id\"\n },\n \"child_conditions\": {\n \"column_name\": \"order_id\"\n }\n }\n ]\n }\n ]\n }\n }\n ```\n" operationId: createNode parameters: - description: Access credentials in the form of a jwt token. in: header name: Authorization required: true type: string - description: ID of the access definition to associate the node with format: uuid in: header name: access-definition-id required: true type: string - description: Type of Node (subset, transform, relationship, insert) enum: - subset - transform - relationship - insert in: path name: node_type required: true type: string - description: Node definition in: body name: body required: true schema: $ref: '#/definitions/node' example: app_data: insert_parameters: relational_integrity: conflict_resolution: upsert name: insert_node table_map: - id: 7e3f4c2a-9b47-4b9c-b2e8-3c81920bd8d1 source: connection_profile: 9c2a77f4-563e-4b07-8a37-6e31a2f30752 database: postgres schema: public source_table_id: 1b82e74d-d849-4f94-a6e2-1f79fe2a5c87 table_name: order_details target: connection_profile: 54e7cd80-7c0b-4a47-87e0-e1b0d47fd2ad database: postgres schema: public table_name: order_details id: 1440b8f4-c7a9-4512-8b51-ad47f4a2108a op: insert_definition type: execution_node produces: - application/json responses: "201": description: Node created successfully examples: application/json: _statusCode_: !!float 201 id: 1440b8f4-c7a9-4512-8b51-ad47f4a2108a message: Node created successfully messageID: OPTTC0701I schema: $ref: '#/definitions/successPostResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 message: 'Validation error occurred: missing required field ''app_data''' messageID: OPTTC0006E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 message: 'Forbidden: User is neither the owner nor an admin; cannot fetch node' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Access definition not found examples: application/json: _statusCode_: !!float 404 message: Record not found. Access definition does not exist messageID: OPTTC0702E schema: $ref: '#/definitions/errorResponse' "409": description: Conflict examples: application/json: _statusCode_: !!float 409 message: Node in use. Node with this ID already exists messageID: OPTTC0703E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Create a new node tags: - Node /v1/node/{node_type}/list: get: description: | Retrieve all nodes associated with a specific node type. operationId: listNodesByType parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token. This is the preferred method of supplying access credentials. in: header name: Authorization required: true type: string - description: Node Type enum: - access - subset - insert - transform - relationship - access_definition - subset_definition - insert_definition - transform_definition - relationship_definition in: path name: node_type required: true type: string produces: - application/json responses: "200": description: Name, ID, Access Definition info, and source connection IDs for all nodes of the specified type examples: application/json: _statusCode_: !!float 200 message: Node List retrieved successfully messageID: OPTTC0707I requestObj: node_type: subset nodes: - access_definition_id: ebe12519-14ad-49ca-a328-7f1c25733a30 access_definition_name: access_node created_at: "2025-11-17T14:51:22.985542Z" created_by_name: User1 Test id: d6149696-f6f1-434c-a6d2-7c09d584ad9b name: subset_node source_connection_ids: - 842ab54c-15d5-4175-a91f-2cc9d681debc schema: $ref: '#/definitions/listNodesByTypeSuccessResponse' "400": description: Bad Request examples: application/json: _statusCode_: !!float 400 message: 'Validation error occurred: invalid node_type value' messageID: OPTTC0006E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 message: 'Forbidden: User is neither the owner nor an admin; cannot fetch node' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: List Nodes by Type tags: - Node /v1/node/{node_type}/{node_id}: delete: description: | Delete a specific Node by its ID. The node_type parameter determines the type of node to delete: - subset: Deletes a subset node with filtering criteria - insert: Deletes an insert node with target table and column mappings - transform: Deletes a transform node with transformation rules - relationship: Deletes a relationship node with entity relationships - access: Deletes an access node operationId: deleteNode parameters: - description: Access credentials in the form of a jwt token. in: header name: Authorization required: true type: string - description: Type of Node (subset, transform, relationship, insert, access) enum: - subset - transform - relationship - insert - access in: path name: node_type required: true type: string - description: UUID of the node in: path name: node_id required: true type: string - description: UUID of the flow in: query name: flow_id type: string produces: - application/json responses: "200": description: Node deleted successfully schema: $ref: '#/definitions/nodeDeleteSuccessResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 message: 'Validation error occurred: invalid node_id format' messageID: OPTTC0006E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 message: 'Forbidden: User is neither the owner nor an admin; cannot fetch node' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Node not found examples: application/json: _statusCode_: !!float 404 message: Record not found. Node does not exist messageID: OPTTC0702E schema: $ref: '#/definitions/errorResponse' "409": description: Conflict examples: application/json: _statusCode_: !!float 409 message: Node in use. Cannot delete node that is in use messageID: OPTTC0703E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Delete Node by ID tags: - Node get: description: | Retrieve a specific Node by its ID. The node_type parameter determines the type of node to retrieve: - subset: Retrieves a subset node with filtering criteria - insert: Retrieves an insert node with target table and column mappings - transform: Retrieves a transform node with transformation rules - relationship: Retrieves a relationship node with entity relationships - access : Retrieves an access node operationId: getNode parameters: - description: Access credentials in the form of a jwt token. in: header name: Authorization required: true type: string - description: Type of Node (subset, transform, relationship, insert, access) enum: - subset - transform - relationship - insert - access in: path name: node_type required: true type: string - description: UUID of the node in: path name: node_id required: true type: string produces: - application/json responses: "200": description: Nodes retrieved successfully schema: $ref: '#/definitions/nodeGetSuccessResponse' "400": description: Invalid parameters examples: application/json: _statusCode_: !!float 400 message: 'Validation error occurred: invalid node_type parameter' messageID: OPTTC0006E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 message: 'Forbidden: User is neither the owner nor an admin; cannot fetch node' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Access definition or Node not found examples: application/json: _statusCode_: !!float 404 message: Record not found. Node does not exist messageID: OPTTC0702E schema: $ref: '#/definitions/errorResponse' "409": description: Conflict examples: application/json: _statusCode_: !!float 409 message: Node in use. Cannot perform operation on node messageID: OPTTC0703E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Get Node by ID tags: - Node /v1/relationship: get: description: | Retrieve relationships the user has access to (creator or admin). Optional filter header narrows results by name or table key substring. operationId: listrelationship parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token in: header name: Authorization required: true type: string - description: 'Optional JSON-encoded array of source tables to filter relationships by parent or child table. Each item: schema, database, table_name, connection_profile.' in: header name: X-Relationship-Filter type: string x-example: '{"source_tables":[{"schema":"DB2INST1","database":"SOURCE","table_name":"DA_CUSTOMERS","connection_profile":"0a5c4d18-3eb8-4a4b-8366-488cde6680bb"},{"schema":"DB2INST1","database":"SOURCE","table_name":"DA_ORDERS","connection_profile":"0a5c4d18-3eb8-4a4b-8366-488cde6680bb"}]}' produces: - application/json responses: "200": description: List of relationship matching the filter criteria examples: application/json: _statusCode_: !!float 200 message: Retrieved relationships successfully messageID: OPTTC0905I requestObj: relationship: - created_at: "2025-11-17T14:47:28.08535Z" created_by: 49ef049d-7a8d-4edf-b41a-4c98b35acce0 created_by_name: User1 Test1 description: Foreign key relationship between customers and orders id: 36e4f6c9-9769-4ff7-b9d3-b2508e0f884e name: CUSTOMERS to ORDERS - created_at: "2025-11-17T15:22:10.12345Z" created_by: 49ef049d-7a8d-4edf-b41a-4c98b35acce0 created_by_name: User1 Test1 description: Foreign key relationship between orders and sales id: a7b2c3d4-e5f6-4a5b-9c8d-7e6f5a4b3c2d name: ORDERS to SALES schema: $ref: '#/definitions/listRelationshipSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: missing required parameter message: 'Core received a bad request: source_tables is required' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: invalid token message: 'Unauthorized access: Token is invalid or has expired' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred attempting to retrieve relationship messageID: OPTTC0906E schema: $ref: '#/definitions/errorResponse' summary: List relationships (get all) tags: - relationship post: consumes: - application/json description: | Create a new relationship between parent and child tables. Can be system-discovered (imported from database metadata) or user-defined. operationId: createRelationship parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token in: header name: Authorization required: true type: string - description: Relationship definition in: body name: body required: true schema: $ref: '#/definitions/relationship' produces: - application/json responses: "201": description: Relationship created successfully examples: application/json: _statusCode_: !!float 201 id: 36e4f6c9-9769-4ff7-b9d3-b2508e0f884e message: Relationship saved successfully messageID: OPTTC0902I schema: $ref: '#/definitions/successPostResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: parent_table is required message: 'Core received a bad request: invalid relationship data' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: invalid token message: 'Unauthorized access: Token is invalid or has expired' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "409": description: Relationship already exists examples: application/json: _statusCode_: !!float 409 errors: [] exception: duplicate relationship message: Relationship already exists messageID: OPTTC0904E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred attempting to save relationship messageID: OPTTC0902E schema: $ref: '#/definitions/errorResponse' summary: Create a new relationship tags: - relationship /v1/relationship/{id}: delete: description: Remove a relationship from the catalog operationId: deleteRelationship parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token in: header name: Authorization required: true type: string - description: The UUID of the relationship in: path name: id required: true type: string x-example: 36e4f6c9-9769-4ff7-b9d3-b2508e0f884e produces: - application/json responses: "200": description: Relationship deleted successfully examples: application/json: _statusCode_: !!float 200 message: Relationship deleted successfully messageID: OPTTC0907I schema: $ref: '#/definitions/successResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid UUID format message: 'Core received a bad request: Invalid relationship ID format' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: invalid token message: 'Unauthorized access: Token is invalid or has expired' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "404": description: Relationship not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: Relationship not found messageID: OPTTC0903E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred attempting to delete relationship messageID: OPTTC0907E schema: $ref: '#/definitions/errorResponse' summary: Delete a relationship tags: - relationship get: description: Retrieve detailed information about a specific relationship operationId: getRelationship parameters: - description: account id in: header name: account-id type: string - description: Access credentials in the form of a jwt token in: header name: Authorization required: true type: string - description: The UUID of the relationship in: path name: id required: true type: string x-example: 36e4f6c9-9769-4ff7-b9d3-b2508e0f884e produces: - application/json responses: "200": description: Relationship details examples: application/json: _statusCode_: !!float 200 message: Retrieved relationship successfully messageID: OPTTC0902I requestObj: created_at: "2026-02-24T18:01:26.107+05:30" created_by: e8ddda98-f485-4793-9538-b0886730f27a created_by_name: User5 Test5 description: Foreign key relationship id: 36e4f6c9-9769-4ff7-b9d3-b2508e0f884e name: SALES to ORDERS relationship: child_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: ORDERS schema: DB2INST1 parent_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: DA_SALES schema: DB2INST1 relationship_conditions: - child_conditions: column_name: SALES_ID parent_conditions: column_name: SALES_ID schema: $ref: '#/definitions/getRelationshipSuccessResponse' "400": description: Bad request examples: application/json: _statusCode_: !!float 400 errors: [] exception: invalid UUID format message: 'Core received a bad request: Invalid relationship ID format' messageID: OPTTC0009E schema: $ref: '#/definitions/errorResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: invalid token message: 'Unauthorized access: Token is invalid or has expired' messageID: OPTTC0513E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden - user does not have access to this relationship examples: application/json: _statusCode_: !!float 403 errors: [] exception: insufficient permissions message: 'Forbidden: User does not have access to this relationship' messageID: OPTTC0514E schema: $ref: '#/definitions/errorResponse' "404": description: Relationship not found examples: application/json: _statusCode_: !!float 404 errors: [] exception: record not found message: Relationship not found messageID: OPTTC0903E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred attempting to retrieve relationship messageID: OPTTC0906E schema: $ref: '#/definitions/errorResponse' summary: Get a specific relationship by ID tags: - relationship /v1/status/livez: get: description: Provides a heartbeat endpoint to check if the server is running and can respond to requests operationId: getLivez responses: "200": description: Success examples: application/json: _statusCode_: !!float 200 message: Core API server up and running messageID: OPTTC0002I schema: $ref: '#/definitions/successResponse' default: description: Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Check API server liveness tags: - Status /v1/status/readyz: get: description: Provides an endpoint to check if the server can connect to the metadb postgresql instance operationId: getReadyz responses: "200": description: Success examples: application/json: _statusCode_: !!float 200 message: 'Core API server up and running metadb connected ts: 2025-07-24 13:04:44.546906 +0000 UTC' messageID: OPTTC0003I schema: $ref: '#/definitions/successResponse' "401": description: Unauthorized examples: application/json: _statusCode_: !!float 401 errors: [] exception: Unauthorized access message: 'Unauthorized access: Token is invalid or has expired. Please authenticate again' messageID: OPTTC0001E schema: $ref: '#/definitions/errorResponse' "403": description: Forbidden examples: application/json: _statusCode_: !!float 403 errors: [] exception: Insufficient permissions message: Access forbidden messageID: OPTTC0004E schema: $ref: '#/definitions/errorResponse' "500": description: Internal Server Error examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' default: description: Error payload examples: application/json: _statusCode_: !!float 500 errors: [] exception: Unexpected error message: Unexpected error occurred messageID: OPTTC0005E schema: $ref: '#/definitions/errorResponse' summary: Check API server readiness tags: - Status definitions: archiveInfo: description: Archive details properties: id: type: string name: type: string type: object authTokenResponse: example: _statusCode_: !!float 201 access_token: ee6tOO1xgafAOqCWfl4WJq7ExR-CrirRhVzTPLryAwZp14uLwvz0vT_8wJpQ9idf3TfnnPDHDJhj1GfB8j3k0zxvnXMcZjtvZwfmog expires_in_sec: !!float 900 message: Auth token retrieved successfully messageID: OPTTC0501I refresh_expires_in_sec: !!float 1800 refresh_token: eyJhbGciOiJIUzUxMiIsInOTczMzFjN2YwIxMDk0NGRhZCJ9.vUje0IfZZcP9Q6GK0vT0Kz1LUl9wW9vu4LXeN5RaJYepwzdsdsf requestObj: null properties: access_token: description: Access token for the user type: string expires_in_sec: format: int32 type: integer refresh_expires_in_sec: format: int32 type: integer refresh_token: description: Refresh token for the user type: string type: object collection: description: Collection details properties: archives: description: List of archives in the collection items: $ref: '#/definitions/archiveInfo' type: array created_at: description: Collection creation timestamp type: string created_by: description: Collection creator ID type: string created_by_name: description: Collection creator name type: string description: description: Collection description type: string id: description: Collection ID type: string last_updated: description: Collection update timestamp type: string name: description: Collection name type: string type: object collectionPayload: description: Collection object example: archives: - b46611c7-9f8e-4bee-8950-e8495e615b12 - d3810226-b0d6-4b45-aaf9-e5c937393c2b description: Collection of archives name: collection-test-1 properties: archives: description: List of archives in the collection items: type: string type: array description: description: Collection description type: string name: description: Collection name minLength: !!float 1 type: string required: - name type: object columnDefinition: properties: comments: description: User provided comments type: string name: description: Name of the column type: string nullable: description: If set to "yes" indicates the column accepts NULL values. If set to "no" indicates the column does not accept NULL values. type: string type_modifier: description: Certain types accept a modifier for further customization. For example, VARCHAR data type accepts the FOR BIT DATA modifier to indicate data should be treated as binary. type: string type_name: description: Data type of the column type: string type_precision: description: Data type precision or length. Only valid for certain data types. format: int32 type: integer type_scale: description: Data type scale. Only valid for certain data types. format: int32 type: integer type_string_codeunit: description: Only valid for string types. It specifies the units of the length attribute which can be OCTETS, CODEUNITS16 or CODEUNITS32. type: string required: - name - type_name - type_precision - type_scale - type_string_codeunit - type_modifier - nullable - comments type: object comparePayload: description: Compare, pipeline-flow-v3 JSON structure for compare operation example: doc_type: pipeline id: 17252f2b-6be1-4824-ba42-3f81c8f36a05 json_schema: https://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json name: compare_sample pipelines: - app_data: ui_data: comments: [] description: compare sample with all properties flow_type: compare worksheet_type: "" id: 53382fb7-1039-4b5c-8c85-704253279ec9 name: compare_sample nodes: - app_data: name: pg_db_source originalId: e8d6ee81-dac4-41f1-8dc3-4e1f1cb47c32 ref: e8d6ee81-dac4-41f1-8dc3-4e1f1cb47c32 connection: name: pg_db_source properties: database: postgres description: test data source host: host1.domain.com password: '{archonenc}poSpj8' port: "5432" ssl: "false" type: postgresql username: srcuser1 ref: e8d6ee81-dac4-41f1-8dc3-4e1f1cb47c32 id: 31bf949f-dc68-4efc-b0aa-165b4779d9c2 op: connection_profile parameters: connection_type: source type: binding - app_data: name: pg_db_target originalId: bc89fbc9-8fc6-4224-a632-fa00687895e0 ref: bc89fbc9-8fc6-4224-a632-fa00687895e0 connection: name: pg_db_target properties: database: postgres description: test data source host: host2.domain.com password: '{archonenc}poSpj8' port: "5432" ssl: "false" type: postgresql username: srcuser2 ref: bc89fbc9-8fc6-4224-a632-fa00687895e0 id: 31bf949f-dc68-4efc-b0aa-165b4779d9c2 op: connection_profile parameters: connection_type: target type: binding - app_data: originalId: 679e014b-a447-499d-b2b0-ec3e3618ecfd source_tables: - connection_profile: e8d6ee81-dac4-41f1-8dc3-4e1f1cb47c32 database: postgres id: 173283ae-9b61-47aa-a0f1-6e53e913d568 schema: public table_name: customers - connection_profile: bc89fbc9-8fc6-4224-a632-fa00687895e0 database: postgres id: 64266592-53de-423c-b834-4eee72e5813d schema: public table_name: customers id: 9cc68b66-1f6c-480b-a412-53eb9f5289c1 op: access_definition type: execution_node - app_data: subset_tables: - database: postgres schema: public source_table_id: 173283ae-9b61-47aa-a0f1-6e53e913d568 subset_parameters: - structured_conditions: clauses: - column_name: customer_id operator: = schema_name: public table_alias: "" table_name: customers value: !!float 1000 condition: AND type: where_clause table_name: customers id: 7eac2d20-588d-4674-b744-05ff62b09261 op: subset_definition type: execution_node - app_data: compare_tables: - originalId: 66be33c2-2222-407d-9cc4-90cb14cefa3a properties: limit: !!float 1000 matchKeys: - customer_id - email offset: !!float 0 orderBy: - column: customer_id direction: ASC - column: created_at direction: DESC tables: original_id: 173283ae-9b61-47aa-a0f1-6e53e913d568 target_id: 64266592-53de-423c-b834-4eee72e5813d id: a6becdf7-19ef-4c5f-bb87-79fd9112371f op: compare_definition type: execution_node runtime_ref: optim-runtime:1.2.0 primary_pipeline: 53382fb7-1039-4b5c-8c85-704253279ec9 schemas: [] version: "3.0" properties: app_data: description: Optim specific fields type: object doc_type: description: pipeline enum: - pipeline type: string id: description: Compare pipeline uuid type: string json_schema: description: Pipeline-flow-v3 schema url enum: - https://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json type: string name: description: Compare pipeline name type: string pipelines: description: Array of pipeline items: $ref: '#/definitions/pipeline' type: array primary_pipeline: description: Compare uuid type: string schemas: items: type: object type: array version: description: Pipeline-flow schema version enum: - "3.0" type: string required: - doc_type - version - primary_pipeline - pipelines type: object compareResponse: description: Compare operation response properties: _statusCode_: example: !!float 201 type: integer message: example: 'Compare response, compareID: 17252f2b-6be1-4824-ba42-3f81c8f36a05, compareName: compare_sample' type: string messageID: example: OPTTC1001I type: string requestObj: description: Response containing comparison results example: comparisonResults: - columns: {} mappingInfo: {} records: [] summary: {} tableInfo: {} type: object type: object connectionProfile: description: Data source/target connection profile example: connection: name: my-postgres-db properties: database: mydb description: Sample postgres connection host: db.example.com password: passsword port: "5432" ssl: "true" ssl_certificate: RsdU5KYmQxMzQvdEpTN1NzVlFlcGo1V3o...== type: postgres username: dbuser properties: connection: description: Connection configuration object properties: name: description: Connection name (user-defined label for this connection) minLength: !!float 1 type: string properties: description: Connection details and authentication configuration type: object ref: description: Connection ID (unique identifier/reference) type: string required: - name - properties type: object type: object connectionProfileDependencyInfo: properties: flows: items: type: object type: array nodes: type: object type: object connectionProfileInfo: description: Data source/target connection profile info example: connection: name: my-snowflake-db properties: database: analytics description: Snowflake analytics warehouse host: snowflake.example.com password: '{archonenc}aVtdtFyKBb5wN4N90Bka7+6yGxNb6ZpZ5PXlzxLSE7+oCj3QOajAIMkq' port: "443" ssl: "false" type: snowflake username: sf_user ref: 987e6543-e21b-12d3-a456-426614174999 last_updated: "2025-07-21T09:51:41.324+05:30" user_id: 885f509-ea59-43fc-b1ad-f050 user_name: User2 Test2 properties: connection: properties: name: description: Connection name (user-defined label for this connection) type: string properties: description: Connection details type: object ref: description: Connection ID (unique identifier/reference) type: string type: object last_updated: description: Timestamp of when the profile was last updated format: time type: string user_id: description: Unique identifier of the profile owner type: string user_name: description: Display name of the profile owner type: string type: object deleteResourceArray: description: An array of IDs of resources to be delete example: ids: - c5fc2d36-c065-4fe3-be28-28f3ac942c71 - f4ef7043-7e6f-4f6c-b413-05053d5e804b - 1b05d025-9682-4dc5-88d6-0088d5c294d5 properties: ids: items: minLength: !!float 1 type: string type: array required: - ids type: object diskPruneRequest: properties: batch_size: default: !!float 1000 description: Number of rows to delete in each batch type: integer dry_run: default: false description: If true, only simulate the operation without deleting data enum: - true - false type: boolean max_table_size_bytes: default: !!float 10000 description: Minimum table size threshold to trigger pruning format: int64 type: integer older_than_days: default: !!float 30 description: Delete records older than this many days type: integer retention_policy: default: AGE_BASED description: | Policy used to determine what data should be pruned. AGE_BASED deletes records older than a threshold. TABLE_DISK_USAGE_BASED prunes when table storage usage exceeds a limit. enum: - AGE_BASED - TABLE_DISK_USAGE_BASED type: string type: object diskPruneSuccessResponse: properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string summary: properties: currentTableDiskUsageBytes: description: Current table disk usage in bytes format: int64 type: integer dryRun: description: Indicates if this was a dry run operation type: boolean maxTableDiskUsageBytes: description: Maximum table size threshold given by user format: int64 type: integer prunedByAge: description: Indicates if records were pruned based on age type: boolean prunedBySize: description: Indicates if records were pruned based on table size type: boolean recordsAffected: description: Number of records deleted (or would be deleted in dry run mode) format: int64 type: integer recordsChecked: description: Total records scanned format: int64 type: integer type: object type: object errorResponse: properties: _statusCode_: description: HTTP status code format: int32 type: integer errors: description: array representation of the error messages items: type: object type: array exception: description: string representation of the error type: string message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string type: object extractJobSuccessResponse: description: Extract, pipeline-flow-v3 document example: _statusCode_: !!float 200 message: 'Getting Extract job for job id: 7cd71244-aac0-4abc-91f6-929d7c19fa18' messageID: OPTTR0135I requestObj: app_data: optim_tdm: version: 1.1.0 doc_type: pipeline id: 7cd71244-aac0-4abc-91f6-929d7c19fa18 pipelines: - app_data: ui_data: comments: [] description: Extract job worksheet_type: Extract id: 27f6644e-5283-4d24-85e2-a816c80088c8 name: test set creation nodes: - connection: name: source_db properties: database: postgres description: source_db desc host: util-cp4d.com port: "5433" ssl: "false" type: postgresql username: postgres ref: 32510432-db76-4ac4-ad4a-39a4f1e348e9 id: a3761753-58cd-40fe-a449-668b0669aa89 inputs: [] op: connection_proExtract outputs: [] parameters: connection_type: source type: binding - connection: name: extract_Extract properties: Extract_type: parquet type: localfs ref: 6abefc55-fcdf-40ef-8ba9-01707eb81272 id: ab5b6b4f-7a91-47a4-b6a1-4cdad3c13396 inputs: [] op: connection_proExtract outputs: [] parameters: connection_type: target type: binding - app_data: source_tables: - connection_proExtract: 32510432-db76-4ac4-ad4a-39a4f1e348e9 database: postgres id: 79eaa985-a398-4e04-8f58-137e28a3edba schema: public table_name: customers - connection_proExtract: 32510432-db76-4ac4-ad4a-39a4f1e348e9 database: postgres id: 6912f124-593f-46ef-aaa7-e953afb9c997 schema: public table_name: products id: 2880a9f4-3c17-4a66-8982-db146e14ddc9 op: access_definition type: execution_node - app_data: mask_tables: - columns: - column_name_source: first_name column_name_target: first_name masking_properties: magen_function: config: {} name: USFirstName type: RandomFabrication masking_definition_id: a61134e9-aacd-4836-872b-dd2d22219305 - column_name_source: last_name column_name_target: last_name masking_properties: magen_function: config: {} name: USLastName type: RandomFabrication masking_definition_id: b1fa8800-ae51-47bd-a7fc-5314ef562075 database: postgres schema: public source_table_id: 79eaa985-a398-4e04-8f58-137e28a3edba table_name: customers id: f727a810-2efc-4160-bd4d-be6ec511dd88 op: transform_definition type: execution_node primary_pipeline: 27f6644e-a816c80088c8 user_id: 49ef049d-4c98b35acce0 user_name: User1 Test version: "3.0" properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/flow' type: object fileJobSuccessResponse: description: File, pipeline-flow-v3 document example: _statusCode_: !!float 200 message: 'Getting archive job for job id: 7cd71244-aac0-4abc-91f6-929d7c19fa18' messageID: OPTTR0135I requestObj: app_data: optim_tdm: version: 1.1.0 doc_type: pipeline id: 7cd71244-aac0-4abc-91f6-929d7c19fa18 pipelines: - app_data: ui_data: comments: [] description: file job worksheet_type: archive id: 27f6644e-5283-4d24-85e2-a816c80088c8 name: test set creation nodes: - connection: name: source_db properties: database: postgres description: source_db desc host: util-cp4d.com port: "5433" ssl: "false" type: postgresql username: postgres ref: 32510432-db76-4ac4-ad4a-39a4f1e348e9 id: a3761753-58cd-40fe-a449-668b0669aa89 inputs: [] op: connection_profile outputs: [] parameters: connection_type: source type: binding - connection: name: extract_file properties: file_type: parquet type: localfs ref: 6abefc55-fcdf-40ef-8ba9-01707eb81272 id: ab5b6b4f-7a91-47a4-b6a1-4cdad3c13396 inputs: [] op: connection_profile outputs: [] parameters: connection_type: target type: binding - app_data: source_tables: - connection_profile: 32510432-db76-4ac4-ad4a-39a4f1e348e9 database: postgres id: 79eaa985-a398-4e04-8f58-137e28a3edba schema: public table_name: customers - connection_profile: 32510432-db76-4ac4-ad4a-39a4f1e348e9 database: postgres id: 6912f124-593f-46ef-aaa7-e953afb9c997 schema: public table_name: products id: 2880a9f4-3c17-4a66-8982-db146e14ddc9 op: access_definition type: execution_node - app_data: mask_tables: - columns: - column_name_source: first_name column_name_target: first_name masking_properties: magen_function: config: {} name: USFirstName type: RandomFabrication masking_definition_id: a61134e9-aacd-4836-872b-dd2d22219305 - column_name_source: last_name column_name_target: last_name masking_properties: magen_function: config: {} name: USLastName type: RandomFabrication masking_definition_id: b1fa8800-ae51-47bd-a7fc-5314ef562075 database: postgres schema: public source_table_id: 79eaa985-a398-4e04-8f58-137e28a3edba table_name: customers id: f727a810-2efc-4160-bd4d-be6ec511dd88 op: transform_definition type: execution_node primary_pipeline: 27f6644e-a816c80088c8 user_id: 49ef049d-4c98b35acce0 user_name: User1 Test version: "3.0" properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/flow' type: object flow: description: Flow, pipeline-flow-v3 document example: app_data: optim_tdm: version: 1.1.0 doc_type: pipeline id: 302e581d-56dd-47dc-9bc2-465c78691a76 json_schema: https://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json parameters: {} pipelines: - app_data: ui_data: comments: [] worksheet_type: subset id: 369d625b-ca3e-45a5-bbd0-08d41c1bc8b8 name: sample-01 nodes: - connection: name: source_connection ref: f4dc15d1-25ee-4a20-800c-caa0d439e451 id: a3761753-58cd-40fe-a449-668b0669aa89 op: connection_profile parameters: connection_type: source type: binding - app_data: source_tables: - connection_profile: f4dc15d1-25ee-4a20-800c-caa0d439e451 database: source_db id: a79a9b95-6e11-4a15-a32e-919ca935228c schema: source_schema table_name: customers id: e4d89905-6952-4e17-b929-3e0917a14091 op: access_definition type: execution_node - app_data: subset_tables: - database: source_db schema: source_schema source_table_id: a79a9b95-6e11-4a15-a32e-919ca935228c subset_parameters: - structured_conditions: clauses: - column_name: customer_id operator: = schema_name: source_schema table_alias: "" table_name: customers value: !!float 1000 - column_name: first_name operator: <> schema_name: source_schema table_alias: "" table_name: customers value: '''EMMA''' condition: AND type: where_clause table_name: customers id: e0e35e2b-af57-4b66-b967-6ef40791ba57 op: subset_definition type: execution_node - connection: name: target_connection ref: 6a7b49eb-4de8-4390-9431-91bfe08fd0b6 id: ab5b6b4f-7a91-47a4-b6a1-4cdad3c13396 op: connection_profile parameters: connection_type: target type: binding - app_data: insert_parameters: relational_integrity: conflict_resolution: merge table_map: - id: 64febd6b-8952-45a2-af49-05c0caec24eb source: connection_profile: f4dc15d1-25ee-4a20-800c-caa0d439e451 database: source_db schema: source_schema source_table_id: a79a9b95-6e11-4a15-a32e-919ca935228c table_name: customers target: connection_profile: 6a7b49eb-4de8-4390-9431-91bfe08fd0b6 database: target_db schema: target_schema table_name: customers id: cf7e971c-a3c3-4643-a4ba-5231dd4c7000 op: insert_definition type: execution_node - app_data: mask_tables: - columns: - column_name_source: first_name column_name_target: first_name masking_properties: magen_function: config: {} name: USFirstName type: FormatPreservingRedactionProcessor masking_definition_id: 982dae75-af7e-457b-9f7f-acda2431df10 database: source_db schema: source_schema source_table_id: a79a9b95-6e11-4a15-a32e-919ca935228c table_name: customers id: daa01f77-aea9-4d9f-a4af-cc4f662b30ac op: transform_definition type: execution_node runtime_ref: optim-runtime:1.0.0 primary_pipeline: 369d625b-ca3e-45a5-bbd0-08d41c1bc8b8 version: "3.0" properties: app_data: description: Optim specific fields type: object doc_type: description: pipeline enum: - pipeline type: string id: description: Pipeline flow document uuid (generated) type: string json_schema: description: Pipeline-flow-v3 schema url enum: - https://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json type: string parameters: description: Optim general parameters type: object pipelines: description: Array of pipeline items: $ref: '#/definitions/pipeline' type: array primary_pipeline: description: Flow uuid type: string version: description: Pipeline-flow schema version enum: - "3.0" type: string required: - id - app_data - doc_type - version - primary_pipeline - pipelines type: object flowCreateUpdateSuccessResponse: description: Flow creation/update response example: _statusCode_: !!float 201 message: 'Flow saved to metadb, ID: a81735c4-a97c-40f9-9c58-d23f09f8c31b, Name: flow-fmt-01' messageID: OPTTC0300I requestObj: id: a81735c4-a97c-40f9-9c58-d23f09f8c31b name: flow-fmt-01 properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: description: Job creation response object properties: id: description: Flow ID type: string name: description: Flow name from pipelines[0].name type: string type: object type: object flowDetails: description: Flow details response properties: description: description: Flow description type: string last_updated: description: Last updated timestamp type: string pipeline: $ref: '#/definitions/flow' user_id: description: Flow owner user ID type: string user_name: description: Flow owner user name type: string type: object flowDetailsSuccessResponse: description: Flow, pipeline-flow-v3 document example: _statusCode_: !!float 200 message: 'Flow details retrieved from metadb, ID: a81735c4-a97c-40f9-9c58-d23f09f8c31b, Name: flow-fmt-01' messageID: OPTTC0322I requestObj: description: flow update test job last_updated: "2025-07-24T11:35:12.569667Z" pipeline: app_data: data_archon: version: 0.1.0 doc_type: pipeline id: b7cd199d-fe8f-4a63-9333-497f022f20ee json_schema: https://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json parameters: {} pipelines: - app_data: ui_data: comments: [] description: flow update test job worksheet_type: mask id: 27f6644e-5283-4d24-85e2-a816c80088c8 name: mvp-fmt-01 nodes: - connection: name: postgres_source_db properties: database: postgres description: postgres_source_db desc host: util-cp4d.com port: "5433" ssl: "false" type: postgresql username: postgres ref: 32510432-db76-4ac4-ad4a-39a4f1e348e9 id: 6ee11bd6-c7de-4473-872c-0f4e86a8bf55 op: connection_profile parameters: connection_type: source type: binding - app_data: source_tables: - connection_profile: 32510432-db76-4ac4-ad4a-39a4f1e348e9 database: postgres id: 79eaa985-a398-4e04-8f58-137e28a3edba schema: public table_name: customers - connection_profile: 32510432-db76-4ac4-ad4a-39a4f1e348e9 database: postgres id: 6912f124-593f-46ef-aaa7-e953afb9c997 schema: public table_name: products id: 2880a9f4-3c17-4a66-8982-db146e14ddc9 op: access_definition type: execution_node - app_data: mask_tables: - columns: - column_name_source: first_name column_name_target: first_name masking_properties: magen_function: config: {} name: USFirstName type: RandomFabrication masking_definition_id: a61134e9-aacd-4836-872b-dd2d22219305 - column_name_source: last_name column_name_target: last_name masking_properties: magen_function: config: {} name: USLastName type: RandomFabrication masking_definition_id: b1fa8800-ae51-47bd-a7fc-5314ef562075 database: postgres schema: public source_table_id: 79eaa985-a398-4e04-8f58-137e28a3edba table_name: customers id: f727a810-2efc-4160-bd4d-be6ec511dd88 op: transform_definition type: execution_node primary_pipeline: 27f6644e-a816c80088c8 version: "3.0" user_id: 49ef049d-4c98b35acce0 user_name: User1 Test properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/flowDetails' type: object flowListMeta: description: Flow metadata properties: access_definition_id: type: string access_definition_name: type: string description: description: Flow description type: string id: description: Flow ID type: string last_updated: type: string name: description: Flow name type: string user_id: description: Created by user ID type: string user_name: description: Created by user name type: string worksheet_type: type: string type: object flowSubmitSuccessResponse: description: Flow submission response example: application/json: _statusCode_: !!float 202 message: Job submitted successfully. messageID: OPTTC0304I requestObj: action: CreateSubmissionResponse serverSparkVersion: 4.0.0 submissionId: driver-20250724120743-0000 success: true properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/sparkSubmissionResponse' type: object getAllCollectionsSuccessResponse: example: _statusCode_: !!float 200 message: Successfully listed created collections messageID: OPTTC0820IE requestObj: - archives: - id: b46611c7-9f8e-4bee-8950-e8495e615b12 name: db2-archive created_at: "2026-03-13T18:52:11.466002+05:30" created_by: 49ef049d-7a8d-4edf-b41a-4c98b35acce0 created_by_name: User1 Test description: Collection of IBM Db2 archives id: ce85b0d8-4139-4a1e-9728-e494e4408249 last_updated: "2026-03-13T18:52:11.466002+05:30" name: db2-collection - archives: - id: b46611c7-9f8e-4bee-8950-e8495e615b12 name: postgres-archive-1 - id: 3373ca45-753c-4bdc-9695-f16123c2d3f9 name: postgres-archive-2 created_at: "2026-03-13T19:26:21.763679+05:30" created_by: 49ef049d-7a8d-4edf-b41a-4c98b35acce0 created_by_name: User1 Test description: Collection of PostgreSQL archives id: fad5d942-2673-4d0a-855e-a070a7afc647 last_updated: "2026-03-13T19:26:21.763679+05:30" name: postgres-collection properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: items: $ref: '#/definitions/collection' type: array type: object getAllConnectionProfilesSuccessResponse: description: Collection of connprofiles example: _statusCode_: !!float 200 message: Retrieved all connection profiles messageID: OPTTC0207I requestObj: infos: - connection: name: my-snowflake-db properties: database: analytics description: Snowflake analytics warehouse host: snowflake.example.com password: '{archonenc}aVtdtFyKBb5wN4N90Bka7+6yGxNb6ZpZ5PXlzxLSE7+oCj3QOajAIMkq' port: "443" ssl: "false" type: snowflake username: sf_user ref: 987e6543-e21b-12d3-a456-426614174999 last_updated: "2025-07-21T09:51:41.324+05:30" user_id: 885f509-ea59-43fc-b1ad-f050 user_name: User2 Test2 - connection: name: my-postgres-db properties: database: mydb description: Production Postgres database host: db.example.com password: '{archonenc}aVtdtFyKBb5wN4N90Bka7-uyGxNb6ZpZ5PXlzxLSk9+oCj3QOajAIMkq' port: "5432" ssl: "true" type: postgres username: dbuser ref: 123e4567-e89b-12d3-a456-426614174000 last_updated: "2025-07-21T09:51:41.324+05:30" user_id: 885f509-ea59-43fc-b1ad-f050 user_name: User2 Test2 properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: properties: infos: items: $ref: '#/definitions/connectionProfileInfo' type: array type: object type: object getArchiveSchemasResponse: description: Get archive schemas success response properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: description: Object with schemas in archive properties: schemas: items: description: Schema details type: object type: array type: object type: object getArchiveTableDataSuccessResponse: description: File table information response example: _statusCode_: !!float 200 message: 'Getting archive table data for job id: 7cd71244-aac0-4abc-91f6-929d7c19fa18, schema: dbo, table: customers' messageID: OPTTC0803I requestObj: columns: - comments: "" name: customer_id nullable: no type_modifier: "" type_name: integer type_precision: !!float 0 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: first_name nullable: yes type_modifier: "" type_name: string type_precision: !!float 0 type_scale: !!float 0 type_string_codeunit: "" name: customers rows: - - "1" - aubrey - dengel - alice@example.com - 123-456-7890 - 123 Maple St - 1225-5708-3529-3084 - visa - "1985-03-12" - "2025-06-05 21:20:34.022267" - - "2" - charlsie - merles - bob@example.com - 987-654-3210 - 456 Oak St - 3994-0104-3898-1368 - visa - "1992-07-25" - "2025-06-05 21:20:34.022267" schema: dbo total_count: !!float 10 properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/tableData' type: object getArchiveTableDetailSuccessResponse: description: File table information response example: _statusCode_: !!float 200 message: 'Getting archive table details for job id: 7cd71244-aac0-4abc-91f6-929d7c19fa18, schema: dbo, table: customers' messageID: OPTTC0801I requestObj: columns: - comments: "" name: customer_id nullable: no type_modifier: "" type_name: integer type_precision: !!float 0 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: first_name nullable: yes type_modifier: "" type_name: string type_precision: !!float 0 type_scale: !!float 0 type_string_codeunit: "" comments: "" conn_profile_name: conn_profile name: customers pk_info: columns: [] comments: Primary key constraint conn_profile_name: conn_profile name: customers_pkey schema_name: dbo table_name: customers schema: dbo stats: last_collected: "2025-10-14T13:00:18.984Z" rows: !!float 5 status: "" properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/tableInfo' type: object getAuthUserinfoSuccessResponse: properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: properties: email: description: User email type: string family_name: description: User last name type: string given_name: description: User first name type: string name: description: User name type: string preferred_username: description: Preferred user name type: string sub: description: User identifier type: string type: object type: object getCollectionSuccessResponse: example: _statusCode_: !!float 200 message: Collection 'db2-collection' retrieved successfully messageID: OPTTC0813I requestObj: archives: - id: b46611c7-9f8e-4bee-8950-e8495e615b12 name: db2-archive created_at: "2026-03-13T18:52:11.466002+05:30" created_by: 49ef049d-7a8d-4edf-b41a-4c98b35acce0 created_by_name: User1 Test description: Collection of IBM Db2 archives id: ce85b0d8-4139-4a1e-9728-e494e4408249 last_updated: "2026-03-13T18:52:11.466002+05:30" name: db2-collection properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/collection' type: object getConfigSuccessResponse: description: Success response for configuration properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: description: Configuration information type: object type: object getConnectionDependenciesSuccessResponse: description: Collection of connprofiles properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/connectionProfileDependencyInfo' type: object getConnectionProfileInfoSuccessResponse: description: Collection of connprofiles example: _statusCode_: !!float 200 message: 'Fetched connection profile: my-postgres-db' messageID: OPTTC0202I requestObj: connection: name: my-postgres-db properties: database: mydb description: Production Postgres database host: db.example.com password: '{archonenc}aVtdtFyKBb5wN4N90Bka7+6yGxNb6ZpZ5PXlzxLSE7+oCj3QOajAIMkq' port: "5432" ssl: "true" type: postgres username: dbuser ref: 123e4567-e89b-12d3-a456-426614174000 last_updated: "2025-07-21T09:51:41.324+05:30" user_id: 885f509-ea59-43fc-b1ad-f050 user_name: User2 Test2 properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/connectionProfileInfo' type: object getExtractSchemasResponse: description: Get Extract schemas success response properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: description: Object with schemas in Extract properties: schemas: items: description: Schema details type: object type: array type: object type: object getExtractTableDataSuccessResponse: description: Extract table information response example: _statusCode_: !!float 200 message: 'Getting Extract table data for job id: 7cd71244-aac0-4abc-91f6-929d7c19fa18, schema: dbo, table: customers' messageID: OPTTC0803I requestObj: columns: - comments: "" name: customer_id nullable: no type_modifier: "" type_name: integer type_precision: !!float 0 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: first_name nullable: yes type_modifier: "" type_name: string type_precision: !!float 0 type_scale: !!float 0 type_string_codeunit: "" name: customers rows: - - "1" - aubrey - dengel - alice@example.com - 123-456-7890 - 123 Maple St - 1225-5708-3529-3084 - visa - "1985-03-12" - "2025-06-05 21:20:34.022267" - - "2" - charlsie - merles - bob@example.com - 987-654-3210 - 456 Oak St - 3994-0104-3898-1368 - visa - "1992-07-25" - "2025-06-05 21:20:34.022267" schema: dbo total_count: !!float 10 properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/tableData' type: object getExtractTableDetailSuccessResponse: description: Extract table information response example: _statusCode_: !!float 200 message: 'Getting extract table details for job:7cd71244-aac0-4abc-91f6-929d7c19fa18, schema: dbo, table: customers' messageID: OPTTC0901I requestObj: columns: - comments: "" name: customer_id nullable: no type_modifier: "" type_name: integer type_precision: !!float 0 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: first_name nullable: yes type_modifier: "" type_name: string type_precision: !!float 0 type_scale: !!float 0 type_string_codeunit: "" comments: "" conn_profile_name: conn_profile name: customers pk_info: columns: [] comments: Primary key constraint conn_profile_name: conn_profile name: customers_pkey schema_name: dbo table_name: customers schema: dbo stats: last_collected: "2025-10-14T13:00:18.984Z" rows: !!float 5 status: "" properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/tableInfo' type: object getRelationshipSuccessResponse: description: Response containing relationship details example: _statusCode_: !!float 200 message: Retrieved relationship successfully messageID: OPTTC0903I requestObj: created_at: "2026-02-24T18:01:26.107+05:30" created_by: e8ddda98-f485-4793-9538-b0886730f27a created_by_name: User5 Test5 description: Foreign key relationship id: 36e4f6c9-9769-4ff7-b9d3-b2508e0f884e name: SALES to ORDERS relationship: child_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: ORDERS schema: DB2INST1 parent_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: DA_SALES schema: DB2INST1 relationship_conditions: - child_conditions: column_name: SALES_ID parent_conditions: column_name: SALES_ID properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/relationshipDetails' type: object getSchemaSuccessResponse: description: Schema information response example: _statusCode_: !!float 200 message: Successfully retrieved schemas from connection profile conn2 messageID: OPTTC0404I requestObj: definer_type: U name: public properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/schema' type: object getSchemaTableByNameSuccessResponse: description: Table information response example: _statusCode_: !!float 200 message: Successfully retreived table definition messageID: OPTTC0401I requestObj: columns: - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: first_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: last_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: email nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 100 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: phone nullable: yes type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 20 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: address nullable: yes type_modifier: TEXT type_name: LONGVARCHAR type_precision: !!float 1073741823 type_scale: !!float 0 type_string_codeunit: "" comments: For the PostgreSQL connector on IBM Cloud, select IBM Cloud Databases for PostgreSQL. conn_profile_name: my_postgres_db name: customers pk_info: columns: - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" comments: Primary key constraint conn_profile_name: my_postgres_db name: customers_pkey schema_name: public table_name: customers schema: public stats: last_collected: "2025-07-24T08:45:59.867Z" rows: !!float 20 properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/tableInfo' type: object getSchemaTablesSuccessResponse: description: Collection of tables in schema example: _statusCode_: !!float 200 message: Successfully listed schema tables messageID: OPTTC0403I requestObj: resources: - name: orders schema: sales - name: customers schema: sales total_count: !!float 2 properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/tables' type: object getSchemasSuccessResponse: description: Collection of schemas example: _statusCode_: !!float 200 message: Successfully retrieved schemas messageID: OPTTC0400I requestObj: resources: - definer_type: U name: public - definer_type: U name: sales total_count: !!float 2 properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/schemas' type: object getSparkRunSuccessResponse: description: Spark logs success response properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/sparkRun' type: object getTableDataSuccessResponse: description: Table data response example: _statusCode_: !!float 200 message: Successfully retreived table data messageID: OPTTC0402I requestObj: columns: - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: first_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: last_name nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 50 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: email nullable: no type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 100 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: phone nullable: yes type_modifier: CHARACTER VARYING type_name: VARCHAR type_precision: !!float 20 type_scale: !!float 0 type_string_codeunit: "" rows: - - "1" - Alice - Johnson - alice@example.com - 123-456-7890 - 123 Maple St - 4111-1111-1111-1111 - visa - "1985-03-12" - "2025-06-05 21:20:34.022267" - - "2" - Bob - Smith - bob@example.com - 987-654-3210 - 456 Oak St - 5500-0000-0000-0004 - visa - "1992-07-25" - "2025-06-05 21:20:34.022267" - - "3" - Charlie - Davis - charlie@example.com - 555-666-7777 - 789 Pine St - 4532-6993-8458-6192 - visa - "1988-05-30" - "2025-06-05 21:20:34.022267" total_count: !!float 3 properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/tableData' type: object getTableDependenciesSuccessResponse: description: Table dependencies response example: _statusCode_: !!float 200 message: Fetched for dependencies for table public.customers messageID: OPTTC0405I requestObj: ancestors: [] conn_profile_name: postgres_source_db depth: !!float 10 descendants: [] direction: ancestors related_tables: [] schema_name: public table_name: customers properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/tableDependenciesInfo' type: object job: description: Job, pipeline-flow-v3 document example: app_data: optim_tdm: version: 1.0.0 doc_type: pipeline id: 302e581d-56dd-47dc-9bc2-465c78691a76 json_schema: https://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json parameters: {} pipelines: - app_data: ui_data: comments: [] worksheet_type: subset id: 369d625b-ca3e-45a5-bbd0-08d41c1bc8b8 name: sample-01 nodes: - connection: name: source_pg_connection ref: f4dc15d1-25ee-4a20-800c-caa0d439e451 id: a3761753-58cd-40fe-a449-668b0669aa89 inputs: [] op: connection_profile outputs: [] parameters: connection_type: source type: binding - app_data: source_tables: - connection_profile: f4dc15d1-25ee-4a20-800c-caa0d439e451 database: source_db id: a79a9b95-6e11-4a15-a32e-919ca935228c schema: source_schema table_name: customers id: e4d89905-6952-4e17-b929-3e0917a14091 op: access_definition type: execution_node - app_data: subset_tables: - database: source_db schema: source_schema source_table_id: a79a9b95-6e11-4a15-a32e-919ca935228c subset_parameters: - structured_conditions: clauses: - column_name: customer_id operator: = schema_name: source_schema table_alias: "" table_name: customers value: !!float 1000 - column_name: first_name operator: <> schema_name: source_schema table_alias: "" table_name: customers value: '''EMMA''' condition: AND type: where_clause table_name: customers id: e0e35e2b-af57-4b66-b967-6ef40791ba57 op: subset_definition type: execution_node - connection: name: target_db2_connection ref: 6a7b49eb-4de8-4390-9431-91bfe08fd0b6 id: ab5b6b4f-7a91-47a4-b6a1-4cdad3c13396 inputs: [] op: connection_profile outputs: [] parameters: connection_type: target type: binding - app_data: insert_parameters: relational_integrity: conflict_resolution: merge table_map: - id: 64febd6b-8952-45a2-af49-05c0caec24eb source: connection_profile: f4dc15d1-25ee-4a20-800c-caa0d439e451 database: source_db schema: source_schema source_table_id: a79a9b95-6e11-4a15-a32e-919ca935228c table_name: customers target: connection_profile: 6a7b49eb-4de8-4390-9431-91bfe08fd0b6 database: target_db schema: target_schema table_name: customers id: cf7e971c-a3c3-4643-a4ba-5231dd4c7000 op: insert_definition type: execution_node - app_data: mask_tables: - columns: - column_name_source: first_name column_name_target: first_name masking_properties: magen_function: config: {} name: USFirstName type: FormatPreservingRedactionProcessor masking_definition_id: 982dae75-af7e-457b-9f7f-acda2431df10 database: source_db schema: source_schema source_table_id: a79a9b95-6e11-4a15-a32e-919ca935228c table_name: customers id: daa01f77-aea9-4d9f-a4af-cc4f662b30ac op: transform_definition type: execution_node runtime_ref: optim-runtime:1.0.0 primary_pipeline: 369d625b-ca3e-45a5-bbd0-08d41c1bc8b8 version: "3.0" properties: app_data: description: Optim specific fields type: object doc_type: description: pipeline enum: - pipeline type: string id: description: Pipeline flow document uuid (generated) type: string json_schema: description: Pipeline-flow-v3 schema url enum: - https://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json type: string parameters: description: Optim general parameters type: object pipelines: description: Array of pipeline items: $ref: '#/definitions/pipeline' type: array primary_pipeline: description: Job uuid type: string version: description: Pipeline-flow schema version enum: - "3.0" type: string required: - doc_type - version - primary_pipeline - pipelines type: object jobDetails: description: Job details response properties: description: description: Job description type: string last_updated: description: Last updated timestamp type: string pipeline: $ref: '#/definitions/job' user_id: description: Job owner user ID type: string user_name: description: Job owner user name type: string type: object jobDetailsSuccessResponse: description: Job, pipeline-flow-v3 document example: _statusCode_: !!float 200 message: 'Job details retrieved successfully, jobID: b7cd199d-fe8f-4a63-9333-497f022f20ee' messageID: OPTTC0301I requestObj: description: flow update test job last_updated: "2025-07-24T11:35:12.569667Z" pipeline: app_data: data_archon: version: 0.1.0 doc_type: pipeline id: b7cd199d-fe8f-4a63-9333-497f022f20ee json_schema: https://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json parameters: {} pipelines: - app_data: ui_data: comments: [] description: flow update test job worksheet_type: mask id: 27f6644e-5283-4d24-85e2-a816c80088c8 name: mvp-fmt-01 nodes: - connection: name: postgres_source_db properties: database: postgres description: postgres_source_db desc host: util-cp4d.com port: "5433" ssl: "false" type: postgresql username: postgres ref: 32510432-db76-4ac4-ad4a-39a4f1e348e9 id: 6ee11bd6-c7de-4473-872c-0f4e86a8bf55 inputs: [] op: connection_profile outputs: [] parameters: connection_type: source type: binding - app_data: source_tables: - connection_profile: 32510432-db76-4ac4-ad4a-39a4f1e348e9 database: postgres id: 79eaa985-a398-4e04-8f58-137e28a3edba schema: public table_name: customers - connection_profile: 32510432-db76-4ac4-ad4a-39a4f1e348e9 database: postgres id: 6912f124-593f-46ef-aaa7-e953afb9c997 schema: public table_name: products id: 2880a9f4-3c17-4a66-8982-db146e14ddc9 op: access_definition type: execution_node - app_data: mask_tables: - columns: - column_name_source: first_name column_name_target: first_name masking_properties: magen_function: config: {} name: USFirstName type: RandomFabrication masking_definition_id: a61134e9-aacd-4836-872b-dd2d22219305 - column_name_source: last_name column_name_target: last_name masking_properties: magen_function: config: {} name: USLastName type: RandomFabrication masking_definition_id: b1fa8800-ae51-47bd-a7fc-5314ef562075 database: postgres schema: public source_table_id: 79eaa985-a398-4e04-8f58-137e28a3edba table_name: customers id: f727a810-2efc-4160-bd4d-be6ec511dd88 op: transform_definition type: execution_node primary_pipeline: 27f6644e-a816c80088c8 version: "3.0" user_id: 49ef049d-4c98b35acce0 user_name: User1 Test properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/jobDetails' type: object jobListSuccessResponse: description: Job list example: application/json: _statusCode_: !!float 200 message: Successfully listed saved jobs messageID: OPTTC0305I requestObj: resources: - created_at: "2025-11-18T10:24:41.123456Z" description: "" flow_id: aa92b15e-77d0-4d1a-b6ad-6d2f122391a3 flow_revision: "3" id: 3f74d5e1-4c59-48e6-9fc8-8c63c319aa11 last_run_id: driver-20251118102442-0000 last_run_status: FINISHED last_run_timestamp: "2025-11-18T10:24:42.504321Z" last_updated: "2025-11-18T10:24:41.123456Z" name: archive_flow_101 source_connection_id: c5b4a923-1f82-4e0e-b5a3-8c9203955f4e source_connection_name: hr_source_db target_connection_id: 7d51b1ad-9d77-4bca-9b77-ced8388f4af2 target_connection_name: archive_file_store user_id: 1e22f4c7-8037-4ea8-99ae-c6b50a2c1134 user_name: Sample User worksheet_type: "" - created_at: "2025-11-18T11:02:55.985734Z" description: "" flow_id: bb22a94f-66af-4b22-93d4-1cc5ef0222bb flow_revision: "4" id: c9f8bb23-6d11-4dd8-8a9b-41b56b5d9988 last_run_id: driver-20251118110256-0001 last_run_status: FAILED last_run_timestamp: "2025-11-18T11:02:56.148299Z" last_updated: "2025-11-18T11:02:55.985734Z" name: archive_flow_102 source_connection_id: af351fd2-9205-4811-93f0-8e76dd13e662 source_connection_name: inventory_db target_connection_id: d34129e3-0ae8-44dd-8a27-7a2ff341be77 target_connection_name: inventory_archive_fs user_id: 1e22f4c7-8037-4ea8-99ae-c6b50a2c1134 user_name: Sample User worksheet_type: "" total_count: !!float 2 properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: description: Array of jobs that saved in metadb type: object type: object killSubmissionSuccessResponse: description: Spark submission kill success response properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/sparkKillResponse' type: object listArchiveFlowsSuccessResponse: description: List saved flows success response properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: description: Array of flows that saved in metadb type: object type: object listExtractFlowsSuccessResponse: description: List saved flows success response properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: description: Array of flows that saved in metadb type: object type: object listMaskingFormatsSuccessResponse: description: Collection of masking format response properties: _statusCode_: type: integer message: type: string messageID: type: string requestObj: properties: formats: items: $ref: '#/definitions/maskingFormatListItem' type: array type: object type: object listMaskingPoliciesSuccessResponse: description: Collection of masking policy response properties: _statusCode_: type: integer message: type: string messageID: type: string requestObj: properties: policies: items: $ref: '#/definitions/maskingPolicyListItemDetails' type: array type: object type: object listNodesByTypeSuccessResponse: description: List of node metadata for the given node type. properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/nodesByType' type: object listRelationshipSuccessResponse: description: Response containing list of relationships example: _statusCode_: !!float 200 message: Retrieved relationships successfully messageID: OPTTC0905I requestObj: relationships: - child_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: ORDERS schema: DB2INST1 description: Foreign key relationship between customers and orders id: 36e4f6c9-9769-4ff7-b9d3-b2508e0f884e name: CUSTOMERS to ORDERS parent_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: CUSTOMERS schema: DB2INST1 relationship_conditions: - child_conditions: column_name: CUSTOMER_ID parent_conditions: column_name: CUSTOMER_ID - child_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: DA_SALES schema: DB2INST1 description: Foreign key relationship between orders and sales id: a7b2c3d4-e5f6-4a5b-9c8d-7e6f5a4b3c2d name: ORDERS to SALES parent_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: ORDERS schema: DB2INST1 relationship_conditions: - child_conditions: column_name: ORDER_ID parent_conditions: column_name: ORDER_ID properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: properties: relationships: description: List of relationships items: $ref: '#/definitions/relationshipSummary' type: array type: object type: object listSavedFlowsSuccessResponse: description: List saved flows success response properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/savedFlows' type: object maskTextPayload: properties: iv: example: b6dd4a74115729165b063d823b0275a85054f6db7d252dd5ff4dd2278e3268a9 type: string key: type: string metadata: example: key1: val1 type: object payload: example: - Albania - Chile items: type: string type: array policy: example: expressions: [] formats: [] graph: edges: [] nodes: - id: FPT root: true processors: - configuration: format: Countries_CaseSensitive mode: tokenize id: FPT loggable: true type: FormatPreservingTokenizationProcessor selectors: [] version: "1.0" type: object required: - policy - payload type: object maskedTextSuccessResponse: properties: _statusCode_: type: integer message: type: string messageID: type: string requestObj: items: type: string type: array type: object maskingCategories: properties: masking_formats: properties: Address: additionalProperties: items: type: string type: array type: object CreditCardInformation: additionalProperties: items: type: string type: array type: object DeviceAndNetworkAddress: additionalProperties: items: type: string type: array type: object Email: additionalProperties: items: type: string type: array type: object FinancialInformation: additionalProperties: items: type: string type: array type: object HealthcareInformation: additionalProperties: items: type: string type: array type: object PersonName: additionalProperties: items: type: string type: array type: object PersonalInformation: additionalProperties: items: type: string type: array type: object type: object pkey_types: items: type: string type: array type: object maskingCategoriesSuccessResponse: properties: _statusCode_: type: integer message: type: string messageID: type: string requestObj: $ref: '#/definitions/maskingCategories' type: object maskingFormat: description: Masking format properties: description: description: Masking format description type: string id: description: Masking format ID type: string name: description: Masking format name minLength: !!float 1 type: string value: minProperties: !!float 1 type: object required: - name - value type: object maskingFormatCreateUpdateSuccessResponse: description: Masking format creation/update response properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: description: Masking format response object properties: id: description: Masking format ID type: string name: description: Masking format name type: string type: object type: object maskingFormatDetails: description: Masking format details properties: format: $ref: '#/definitions/maskingFormat' last_updated: description: Last updated timestamp type: string user_id: description: Created by user ID type: string user_name: description: Created by user name type: string type: object maskingFormatListItem: description: Masking format list item properties: description: description: Masking format description type: string id: description: Masking format ID type: string last_updated: description: Last updated timestamp type: string name: description: Masking format name type: string user_id: description: Created by user ID type: string user_name: description: Created by user name type: string type: object maskingFormatSuccessResponse: description: Masking format response properties: _statusCode_: type: integer message: type: string messageID: type: string requestObj: $ref: '#/definitions/maskingFormatDetails' type: object maskingPolicy: description: Masking policy properties: description: description: Masking policy description type: string id: description: Masking policy ID type: string locked: description: System-defined built-in functions that are immutable type: boolean name: description: Masking policy name type: string type: description: 'Function execution location eg: optim_policy, runtime, lookup' type: string value: type: object required: - name - locked - type - value type: object maskingPolicyCreateUpdateSuccessResponse: description: Masking policy creation/update response properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: description: Masking policy response object properties: id: description: Masking policy ID type: string name: description: Masking policy name type: string type: object type: object maskingPolicyDetails: description: Masking policy details properties: last_updated: description: Last updated timestamp type: string policy: $ref: '#/definitions/maskingPolicy' user_id: description: Created by user ID type: string user_name: description: Created by user name type: string type: object maskingPolicyListItem: description: Masking policy list item properties: description: description: Masking policy description type: string id: description: Masking policy ID type: string locked: description: System-defined built-in functions that are immutable type: boolean name: description: Masking policy name type: string type: description: 'Function execution location eg: optim_policy, runtime, lookup' type: string required: - name - locked - type type: object maskingPolicyListItemDetails: description: Masking policy list item details properties: last_updated: description: Last updated timestamp type: string policy: $ref: '#/definitions/maskingPolicyListItem' user_id: description: Created by user ID type: string user_name: description: Created by user name type: string type: object maskingPolicySuccessResponse: description: Masking policy response properties: _statusCode_: type: integer message: type: string messageID: type: string requestObj: $ref: '#/definitions/maskingPolicyDetails' type: object maskingTypes: properties: masking_formats: properties: Address: items: type: string type: array Credit_card_information: items: type: string type: array Device_and_network_address: items: type: string type: array Email: items: type: string type: array Financial_information: items: type: string type: array Healthcare_information: items: type: string type: array Person_Name: items: type: string type: array Personal_information: items: type: string type: array type: object pkey_types: items: type: string type: array types: items: type: string type: array type: object maskingTypesSuccessResponse: example: _statusCode_: !!float 200 message: Successfully retreived masking formats and types messageID: OPTTC0601I requestObj: masking_formats: Address: - Address - City - State - Zip Device_and_network_address: - IP - Ipv4 - Ipv6 - MacAddress Person_Name: - First pkey_types: - RepeatableFormatFabricationProcessor - FormatPreservingTokenizationProcessor - RepeatableNoiseProcessor types: - FormatPreservingRedactionProcessor - RandomFormatFabricationProcessor - RepeatableFormatFabricationProcessor - FormatPreservingTokenizationProcessor - RandomNoiseProcessor - RepeatableNoiseProcessor properties: _statusCode_: type: integer message: type: string messageID: type: string requestObj: $ref: '#/definitions/maskingTypes' type: object node: description: Node request definition example: app_data: name: sales-def-12 source_tables: - connection_profile: 348c7b00-972d-4e59-90c2-c230ac8b9d41 database: mysql id: 91f32c4e-8ac1-4c2a-9ce2-11ab5b8937de schema: retail table_name: orders - connection_profile: 348c7b00-972d-4e59-90c2-c230ac8b9d41 database: mysql id: 27abc7dd-4fd3-4ec8-85fe-4f995b3b00a8 schema: retail table_name: payments id: 9694c125-4f7f-4e3d-80a9-661df0a1c2be op: access_definition type: execution_node properties: app_data: additionalProperties: true description: Application-specific data for the node that varies based on node type type: object id: description: uuid for the node type: string op: description: Operation type (e.g., subset_definition) enum: - subset_definition - insert_definition - transform_definition - relationship_definition - access_definition type: string type: description: Node type (e.g., execution_node) enum: - execution_node - binding type: string required: - id - op - type - app_data type: object nodeDeleteSuccessResponse: description: Node response example: _statusCode_: !!float 200 id: 7252f00e-5d8c-45a7-b8c6-ac3807f44a81 message: Node deleted successfully messageID: OPTTC0701I properties: _statusCode_: description: HTTP status code format: int32 type: integer id: description: Node ID type: string message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string type: object nodeGetSuccessResponse: description: Node response example: _statusCode_: !!float 201 message: Node retrieved successfully messageID: OPTTC0705I requestObj: app_data: subset_tables: - database: postgres schema: source_schema source_table_id: 79eaa985-a398-4e04-8f58-137e28a3edba subset_parameters: - structured_conditions: clauses: - column_name: customer_id operator: = schema_name: source_schema table_alias: "" table_name: customers value: !!float 1000 - column_name: first_name operator: <> schema_name: source_schema table_alias: "" table_name: customers value: '''EMMA''' condition: AND type: where_clause table_name: customers properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: type: object type: object nodeListByAccessDefinition: description: Response containing nodes related to an access definition example: _statusCode_: !!float 200 message: Node List retrieved successfully messageID: OPTTC0707I requestObj: access_definition_id: 550e8400-e29b-41d4-a716-446655440000 related_nodes: insert_definition: - id: c4a81735-97c4-9c58-40f9-f8c31bd23f09 name: insert 1 relationship_definition: - id: 40f9c4a8-1735-97c4-9c58-d23f09f8c31b name: relationship 1 subset_definition: - id: 7252f00e-5d8c-45a7-b8c6-ac3807f44a91 name: subset 1 transform_definition: - id: a81735c4-a97c-40f9-9c58-d23f09f8c31b name: transform 1 properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: properties: access_definition_id: description: UUID of the access definition type: string related_nodes: properties: insert_definition: items: $ref: '#/definitions/nodeSummary' type: array relationship_definition: items: $ref: '#/definitions/nodeSummary' type: array subset_definition: items: $ref: '#/definitions/nodeSummary' type: array transform_definition: items: $ref: '#/definitions/nodeSummary' type: array type: object type: object type: object nodeMeta: description: Node metadata properties: access_definition_id: default: "" description: Related access definition ID type: string access_definition_name: default: "" description: Related access definition name type: string archive_ids: description: List of archive IDs, for the related access definition items: type: string type: array x-omitempty: true created_at: default: "" description: Created at timestamp format: time type: string created_by_name: default: "" description: Created by user name type: string extract_ids: description: List of extract IDs, for the related access definition items: type: string type: array x-omitempty: true id: description: Node ID type: string name: default: "" description: Node name type: string source_connection_ids: description: List of source connection IDs, for the related access definition items: type: string type: array required: - id - name - access_definition_id - access_definition_name - source_connection_ids - created_by_name - created_at type: object nodeSummary: description: Summary information about a node properties: id: description: UUID of the node type: string name: description: Name of the node type: string required: - id - name type: object nodesByType: description: Collection of node metadata for the given node type properties: node_type: description: The node type type: string nodes: description: List of node metadata items: $ref: '#/definitions/nodeMeta' type: array required: - node_type - nodes type: object pipeline: description: Pipeline-flow object that contains a job properties: app_data: description: Optim specific fields properties: ui_data: description: Fields for additonal ui info properties: comments: items: type: string type: array description: type: string flow_type: description: Flow type set by ui for filtering type: string worksheet_type: description: Worksheet type set by ui for filtering type: string type: object type: object id: description: Job uuid type: string name: description: Job name (unique) type: string nodes: description: Array of nodes that make up the Job actions and connections items: type: object type: array runtime_ref: description: Runtime id example: optim-runtime:0.1.0 type: string required: - id - name - app_data type: object previewRows: description: Sample row response based on subsetting and masking inputs properties: resources: description: A list of table data object items: $ref: '#/definitions/tableData' type: array total_count: description: Number of elements format: int32 type: integer required: - resources - total_count type: object previewRowsSuccessResponse: description: Sample rows response example: _statusCode_: !!float 200 id: 0dfb8b8b-fc93-42d1-b193-3d7d8c92b861 message: 'Sample row retrieval successful, jobID: 0dfb8b8b-fc93-42d1-b193-3d7d8c92b861' messageID: OPTTC0310I requestObj: resources: - columns: - comments: "" name: order_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: customer_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: total_amount nullable: yes type_modifier: NUMERIC type_name: NUMERIC type_precision: !!float 10 type_scale: !!float 2 type_string_codeunit: "" name: orders rows: - - "1" - "1" - "2025-05-01 10:00:00.0" - Completed - "1400.00" - - "2" - "2" - "2025-05-02 12:30:00.0" - Pending - "350.00" - - "3" - "3" - "2025-05-03 15:45:00.0" - Shipped - "1200.00" schema: public total_count: !!float 3 - columns: - comments: "" name: payment_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: order_id nullable: no type_modifier: INTEGER type_name: INTEGER type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: "" name: payment_date nullable: yes type_modifier: TIMESTAMP type_name: TIMESTAMP type_precision: !!float 26 type_scale: !!float 6 type_string_codeunit: "" - comments: "" name: amount nullable: no type_modifier: NUMERIC type_name: NUMERIC type_precision: !!float 10 type_scale: !!float 2 type_string_codeunit: "" name: payments rows: - - "1" - "1" - "2025-05-01 10:30:00.0" - "1400.00" - - "2" - "2" - "2025-05-02 12:45:00.0" - "350.00" - - "3" - "3" - "2025-05-03 16:00:00.0" - "1200.00" schema: public total_count: !!float 3 total_count: !!float 2 properties: _statusCode_: description: HTTP status code format: int32 type: integer id: description: Job identifier type: string message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/previewRows' type: object primaryKeyInfo: description: Table primary key info example: columns: - comments: Primary key name: order_id nullable: no type_modifier: "" type_name: integer type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" comments: Primary key for orders conn_profile_name: conn1 name: orders_pk schema_name: sales table_name: orders properties: columns: description: table primary key column info, returns empty array if primary key is not defined items: $ref: '#/definitions/columnDefinition' type: array comments: description: User provided comments type: string conn_profile_name: description: connection profile name type: string name: description: The primary key name type: string schema_name: description: The schema name type: string table_name: description: The table name type: string required: - conn_profile_name - name - schema_name - table_name - columns - comments type: object processReportSuccessResponse: description: Successful process report retrieval properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: description: Process report data from runtime type: object type: object relationship: description: Relationship between parent and child tables (POST body and GET response use this flat shape) example: child_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: ORDERS schema: DB2INST1 created_at: "2026-02-24T18:01:26.107+05:30" created_by: e8ddda98-f485-4793-9538-b0886730f27a created_by_name: User5 Test5 description: Foreign key relationship between sales and orders tables id: 36e4f6c9-9769-4ff7-b9d3-b2508e0f884e name: SALES to ORDERS parent_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: DA_SALES schema: DB2INST1 relationship_conditions: - child_conditions: column_name: SALES_ID parent_conditions: column_name: SALES_ID properties: child_table: $ref: '#/definitions/tableReference' created_at: description: Creation timestamp format: date-time type: string created_by: description: UUID of user who created the relationship type: string created_by_name: description: Name of user who created the relationship type: string description: description: Description of the relationship type: string id: description: UUID of the relationship (optional - generated if not provided) type: string name: description: Logical name of the relationship minLength: !!float 1 type: string parent_table: $ref: '#/definitions/tableReference' relationship_conditions: description: List of join conditions items: $ref: '#/definitions/relationshipCondition' type: array required: - name - parent_table - child_table - relationship_conditions type: object relationshipCondition: description: Join condition between parent and child tables example: child_conditions: column_name: SALES_ID parent_conditions: column_name: SALES_ID properties: child_conditions: properties: column_name: description: Column name in child table type: string required: - column_name type: object parent_conditions: properties: column_name: description: Column name in parent table type: string required: - column_name type: object required: - parent_conditions - child_conditions type: object relationshipDetails: description: Relationship get-by-id response with value (JSONB payload) wrapped separately from metadata example: created_at: "2026-02-24T18:01:26.107+05:30" created_by: e8ddda98-f485-4793-9538-b0886730f27a created_by_name: User5 Test5 description: Foreign key relationship id: 36e4f6c9-9769-4ff7-b9d3-b2508e0f884e name: SALES to ORDERS relationship: child_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: ORDERS schema: DB2INST1 parent_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: DA_SALES schema: DB2INST1 relationship_conditions: - child_conditions: column_name: SALES_ID parent_conditions: column_name: SALES_ID properties: created_at: description: Creation timestamp format: date-time type: string created_by: description: UUID of user who created the relationship type: string created_by_name: description: Name of user who created the relationship type: string description: description: Description of the relationship type: string id: description: UUID of the relationship type: string name: description: Logical name of the relationship minLength: !!float 1 type: string relationship: $ref: '#/definitions/relationshipValue' required: - id - name - relationship type: object relationshipSummary: description: Summary information about a relationship (includes full relationship details) example: child_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: ORDERS schema: DB2INST1 child_table_name: ORDERS created_at: "2025-11-17T14:47:28.08535Z" created_by: 49ef049d-7a8d-4edf-b41a-4c98b35acce0 created_by_name: User1 Test1 description: Foreign key relationship id: 36e4f6c9-9769-4ff7-b9d3-b2508e0f884e name: SALES to ORDERS parent_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: DA_SALES schema: DB2INST1 parent_table_name: DA_SALES relationship_conditions: - child_conditions: column_name: SALES_ID parent_conditions: column_name: SALES_ID properties: child_table: $ref: '#/definitions/tableReference' child_table_name: description: Name of the child table type: string created_at: description: Creation timestamp format: date-time type: string created_by: description: UUID of user who created the relationship type: string created_by_name: description: Name of user who created the relationship type: string description: description: Description of the relationship type: string id: description: UUID of the relationship type: string name: description: Logical name of the relationship type: string parent_table: $ref: '#/definitions/tableReference' parent_table_name: description: Name of the parent table type: string relationship_conditions: description: List of join conditions items: $ref: '#/definitions/relationshipCondition' type: array required: - id - name - parent_table - child_table - relationship_conditions type: object relationshipValue: description: Complete relationship metadata stored in JSONB example: child_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: ORDERS schema: DB2INST1 parent_table: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: DA_SALES schema: DB2INST1 relationship_conditions: - child_conditions: column_name: SALES_ID parent_conditions: column_name: SALES_ID properties: child_table: $ref: '#/definitions/tableReference' parent_table: $ref: '#/definitions/tableReference' relationship_conditions: description: List of join conditions items: $ref: '#/definitions/relationshipCondition' type: array required: - parent_table - child_table - relationship_conditions type: object savedFlows: description: Collection of flow metadata properties: resources: items: $ref: '#/definitions/flowListMeta' type: array total_count: description: Total number of flows returned by the current request type: integer required: - resources - total_count type: object schema: example: definer_type: U name: public properties: definer_type: description: S = The definer is the system; U = The definer is an individual user type: string name: description: The schema name type: string required: - name type: object schemas: description: Collection of schemas properties: resources: description: List of schemas items: $ref: '#/definitions/schema' type: array total_count: description: Number of elements format: int64 type: integer required: - resources - total_count type: object sparkKillResponse: description: Spark response to submit job/job and status queries example: action: KillSubmissionResponse message: Kill request for driver-20151008145126-0000 submitted serverSparkVersion: 4.0.1 submissionId: driver-20151008145126-0000 success: "true" properties: action: description: Spark submission action type: string message: description: message type: string serverSparkVersion: type: string submissionId: description: Spark id for retrieving status type: string success: type: string workerHostPort: type: string workerId: type: string type: object sparkRun: description: Spark run details properties: duration_ms: description: Run duration milliseconds format: int32 type: integer end_time: description: Run end datetime type: string spark_app_id: description: Spark application ID type: string start_time: description: Run start datetime type: string status: description: Spark driver status type: string std_err_log: description: Run STDERR log type: string std_out_log: description: Run STDOUT log type: string submission_id: description: Spark submission ID type: string type: object sparkRunStatus: description: Sample row response based on subsetting and masking inputs properties: job_id: description: Job ID type: string spark_runs: description: A list of spark runs items: $ref: '#/definitions/sparkRun' type: array required: - job_id - spark_runs type: object sparkRunStatusSuccessResponse: description: Status list example: _statusCode_: !!float 200 message: 'Run statuses retrieved successfully for jobID: b9f4f675-f744-4641-b323-401b9f587a7a' messageID: OPTTC0311I requestObj: job_id: b9f4f675-f744-4641-b323-401b9f587a7a spark_runs: - duration_ms: !!float 53337 end_time: 2025-07-24 09:30:32.564 -0700 PDT start_time: 2025-07-24 09:29:32.441425 -0700 PDT status: FINISHED std_err_log: std_out_log: submission_id: driver-20250724162932-0005 properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/sparkRunStatus' type: object sparkSubmissionResponse: description: Spark response to submit job/job and status queries properties: action: description: Spark submission action type: string driverState: description: Status value type: string serverSparkVersion: type: string submissionId: description: Spark id for retrieving status type: string success: type: boolean workerHostPort: type: string workerId: type: string type: object successAuthRefreshTokenResponse: properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/authTokenResponse' type: object successAuthTokenResponse: properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string requestObj: $ref: '#/definitions/authTokenResponse' type: object successPostResponse: properties: _statusCode_: description: HTTP status code format: int32 type: integer id: description: id of the added object type: string message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string type: object successResponse: properties: _statusCode_: description: HTTP status code format: int32 type: integer message: description: Common logger message text type: string messageID: description: Common logger message identifier type: string type: object tableData: example: columns: - comments: Primary key name: order_id nullable: no type_modifier: "" type_name: integer type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: Foreign key to customer name: customer_id nullable: no type_modifier: "" type_name: integer type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" name: orders rows: - - "1" - "100" - - "2" - "101" schema: sales total_count: !!float 2 properties: columns: description: Column definitions items: $ref: '#/definitions/columnDefinition' type: array name: description: Table name type: string rows: description: Table rows items: items: type: string type: array type: array schema: description: Schema name type: string total_count: description: Number of rows format: int64 type: integer type: object tableDependenciesInfo: description: Table dependency info example: ancestors: - name: customers schema: sales conn_profile_name: conn1 depth: !!float 2 descendants: - name: order_items schema: sales direction: ancestors related_tables: - name: customers schema: sales - name: order_items schema: sales schema_name: sales table_name: orders properties: ancestors: description: ancestor tables that are sorted from oldest items: $ref: '#/definitions/tableSummary' type: array conn_profile_name: description: connection profile name type: string depth: description: The level of dependencies type: integer descendants: description: descendant tables that are sorted from oldest items: $ref: '#/definitions/tableSummary' type: array direction: description: direction of the traversal type: string related_tables: description: related tables (ancestors and descendents) items: $ref: '#/definitions/tableSummary' type: array schema_name: description: The schema name type: string table_name: description: The start table name type: string required: - conn_profile_name - schema_name - table_name - direction - depth type: object tableInfo: description: Includes table definition and statistics. Note that statistics such as row count and size are not updated real time. Check the stats_timestamp value to know how current that information is. example: columns: - comments: Primary key name: order_id nullable: no type_modifier: "" type_name: integer type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" - comments: Foreign key to customer name: customer_id nullable: no type_modifier: "" type_name: integer type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" comments: Order table conn_profile_name: conn1 name: orders pk_info: columns: - comments: Primary key name: order_id nullable: no type_modifier: "" type_name: integer type_precision: !!float 10 type_scale: !!float 0 type_string_codeunit: "" comments: Primary key for orders conn_profile_name: conn1 name: orders_pk schema_name: sales table_name: orders schema: sales stats: last_collected: "2024-06-01T12:00:00.000Z" rows: !!float 1000 status: active properties: columns: description: table columns items: $ref: '#/definitions/columnDefinition' type: array comments: description: User provided comments type: string conn_profile_name: description: connection profile name type: string name: description: The table name type: string pk_info: $ref: '#/definitions/primaryKeyInfo' schema: description: The schema name type: string stats: properties: last_collected: description: Timestamp when statistics were last collected for the table. It can be empty for newly created tables. format: YYYY-MM-DDTHH:mm:ss.sssZ type: string rows: description: Number of rows in the table format: int64 type: integer required: - last_collected - rows type: object status: description: It can assume values type: string required: - conn_profile_name - schema - name - columns - comments - stats type: object tableReference: description: Reference to a database table example: connection_profile: 19eee91a-a857-417c-88b1-70af7f071534 database: db2 source name: DA_SALES schema: DB2INST1 properties: connection_profile: description: UUID of the connection profile minLength: !!float 1 type: string database: description: Database name minLength: !!float 1 type: string name: description: Table name minLength: !!float 1 type: string schema: description: Schema name minLength: !!float 1 type: string required: - connection_profile - database - schema - name type: object tableSummary: properties: name: description: The table name type: string schema: description: The schema name type: string required: - name - schema type: object tables: description: Collection of tables properties: resources: description: List of tables items: $ref: '#/definitions/tableSummary' type: array total_count: description: Number of elements format: int64 type: integer required: - resources - total_count type: object tags: - description: View and query archive files within the system name: Archive - description: Authentication APIs name: Auth - description: Organize and manage archive files by grouping them into logical collections name: Collection - description: Compare operations for data comparison name: Compare - description: Product configuration name: Config - description: Manage connection profiles for databases name: Connection Profile - description: APIs to list database schemas, tables, and related objects name: Database Objects - description: APIs to view, manage, and browse extracted data files within the application name: Extract - description: APIs to manage flows within the application name: Flow - description: APIs to view job submissions name: Job - description: System maintenance and cleanup operations name: Maintenance - description: APIs for predefined masking policies and custom formats name: Masking - description: APIs for managing flow nodes name: Node - description: Create logical links between database tables to preserve referential integrity during extraction name: relationship - description: System health check APIs name: Status