Data file loading
To load account or meter data, you configure a file loading data service and associate it with a file loading data pipeline.
System pipelines are automatically created and connected to all enabled file loading data services. You need to create a custom pipeline only if you are using a connector for file transformation or for other special file handling scenarios. For more information, see Provisioning data pipelines.
The end-to-end flow is as follows:
- Create a data service under the category File Loading:
- To load data from an Amazon S3 bucket, set the data service type to Amazon S3 (Cloud Storage). For more information, refer to Amazon S3 data service.
- To load data from Amazon SFTP, set the data service type to Amazon Transfer (SFTP). For more information, refer to Amazon SFTP data service.
- To load data from an Amazon mailbox, set the data service type to Amazon Simple Email (Mailbox). For more information, refer to Amazon mailbox service.
- Optionally, configure a custom file loading pipeline:
- Specify accounts or meters as the target system.
- Specify a filename pattern to use when looking for files to load.
- Select the data services to poll.
- Select none (pass-through) as the data transformer.