Data file transformation and loading

Edit online

Transform and load account or meter data files that are formatted in a client or supplier specific format.

Note: A Connector AddOn is required for data transformation. For more information, see the Connectors section.
To transform account or meter data in App Connect, create both a file loading data service and an App Connect transformation data service, and associate both with a file loading data pipeline.
The end-to-end flow is as follows:
  1. Create a data service under the category File Loading.
    • To load data from an Amazon S3 bucket, set the data service type to Amazon S3 (Cloud Storage). For more information, refer to Amazon S3 data service.
    • To load data from Amazon SFTP, set the data service type to Amazon Transfer (SFTP). For more information, refer to Amazon SFTP data service.
    • To load data from an Amazon mailbox, set the data service type to Amazon Simple Email (Mailbox). For more information, refer to Amazon mailbox service.
  2. Create an App Connect data service under the Transformation category. For more information, refer to Provisioning IBM App Connect.
  3. Configure a file loading pipeline.
    1. Specify accounts or meters as the target system.
    2. Specify a filename pattern to use when looking for files to load.
    3. Select the data services to poll.
    4. Select the App Connect Data ETL service as the data transformer.
    5. Specify any additional parameters that are required by App Connect.
    For detailed see Configuring a file transformation and loading pipeline.