Defining a target

Edit online

The final step in configuring an IBM® Optim Data Privacy flow is defining the target for the extracted data. Targets can be connected databases or extract files.

Before you begin

Ensure that you have created your target database connection in the Connectivity interface. The target database and schema must already exist.

Important: When you configure a target in the IBM Optim UI, the schema selection list is populated from schemas that already exist in the target database, so the prerequisite is enforced automatically. If you submit a flow by using the REST API with a manually drafted JSON payload, you must verify that the schema you specify in the payload exists in the target database before you submit the job. On databases that support implicit schema creation, such as IBM Db2®, a non-existent schema may be created automatically by the database, which deviates from the expected behavior. For more information, see IBM Db2 may implicitly create a missing target schema when a flow is submitted via the API.

For information about creating connections, see Creating a connection.

Adding a target to a flow

You can define the target for your flow by adding a Target node onto a flow builder canvas. Target nodes must follow an Access definition, Subset, or Mask node in a flow.

A flow can have multiple target nodes. Each target node is configured independently through a tearsheet that collects all target details before the node is added to the canvas. You can connect multiple target nodes to the same upstream node to send data to more than one destination in a single flow run.

Figure 1. A target is the final step in the flow of data in IBM Optim
Data flows from the source, through the access definition, then undergoes subsetting and masking before going to the target
To add a target to a flow:
  1. From the navigation menu, select Workflow icon Workflow > Flow builder, then open a flow builder canvas:
    • Click New flow to start a new flow builder canvas.
      or
    • Load an existing flow from the My flows panel.
      or
    • Click Import to load a previously exported flow from a valid JSON flow definition. For more information, see Importing and exporting a flow.
      or
    • Click Node templates to view templates saved from existing nodes such as access definitions, subsets, masking policies, or targets.
  2. Add an access definition node to your flow builder canvas, if one does not already exist. For more information, see Creating an access definition.
  3. Optionally, create subset and mask nodes. For more information, see Applying subsetting conditions and Applying subsetting conditions.
  4. Connect a target node to the previous access definition, subset, or mask node in your flow. You can either create a new target or add a predefined target from your node templates.
    • To create a new target:
      1. Click on the circle on the side of the previous node in the flow, then drag the connector line to an open position on your canvas. Select New target.
      2. Choose whether the new target is a New target connection or a New extract file.
    • To add a target based on an existing node template, drag a target template from the Node templates panel and drop it onto the canvas. For more information, see Creating and working with template nodes.
  5. Define your target node details:
    To define a target connection:
    1. Provide a name and description for the target connection. Click Next.
    2. Click Add insert definition, then specify the target database, schema, and tables.
    3. Select the source tables to send to the target as part of this insert definition. Optionally, edit the names of the target tables. Click Save.
      Important: Include every source table from the Access Definition in this insert definition. If you omit a source table, the runtime uses the source schema name and the first available target connection for that table. If the source schema does not exist in the target database, the job fails at runtime. To add missing tables later, right-click the Insert definition node in the flow builder and select Edit. For more information, see Source tables omitted from the Insert Definition cause runtime to apply unsafe defaults.
    4. Optionally, create additional insert definitions to define more than one target database. You cannot send the same source table to multiple target databases. Once a source table is included in an insert definition, it cannot be included in others.
    5. Click Next.
    6. Configure how the data will be added to the target database:
      • Upsert: New records are inserted into the database and existing records are updated. If the table does not exist, it will be created.
      • Insert only: Records are inserted into the database, but existing records are not updated. If the table does not exist, it will be created.
      • Update only: New records will not be inserted into the database. Only existing records are updated. If the table does not exist, it will be created.
    7. Confirm that you understand that the target database will be altered to add any missing tables. Click Create.
    To define a target extract file:

    Start by defining whether this target will create a new extract file or use an existing extract file.

    You can only use existing extract files that have been configured to allow data to be overwritten. If none are configured, you must create a new extract file.

    • To select an existing extract file to use with this target:
      1. Choose Use an existing extract file.
      2. Choose an extract file from the list of available files.
      3. Click Create.
    • To create a new extract file to use with this target:
      1. Choose Create new extract file.
      2. Provide a name and description for the new extract file target.
      3. Configure whether this extract file can be overwritten. When selected, this extract file can be written to more than once, including potentially from other workflows in the system.
        Important: File data that is overwritten cannot be recovered.
      4. Configure whether the extract file will be encrypted.
      5. Select a storage profile to set the location where the file will be stored. For information about setting up storage locations, see Managing storage profiles for extraction.
      6. Configure the compression type used on this extract file.
      7. Click Create.