Provisioning data pipelines

Edit online

In IBM® Envizi ESG Suite, create custom data pipelines to load files into the platform or export data from the platform.

System pipelines are automatically created and connected to all enabled file loading data services and do not require configuration. However, custom pipelines are required for connectors, including both legacy connectors and new connectors that go through IBM App Connect. Custom pipelines are also required for API integrations and if you want to apply special handling when bulk uploading templates.

Note: Use any S3 buckets that you create through IBM Envizi ESG Suite only for the purpose of uploading data to the platform.
When files are uploaded the following checks are done:
  1. First, the filenames are checked against user-defined custom pipelines. If the file matches only one pipeline, it is processed by that pipeline.
  2. If the filenames do not match any user-defined custom pipeline filename patterns, the files are checked against system pipeline filename patterns. If the file matches only one system pipeline, it is processed by that system pipeline.
  3. If the filenames do not match any system pipeline filename patterns, one of the following scenarios applies:
    • If the files are uploaded through the Upload Files page, the files are assumed to contain account data and are checked against any legacy account data loading file specifications.
    • If the files are uploaded through a data flow automation service, the files are moved to an error folder and are not processed.