Configuring an export pipeline

Configure a pipeline to export monthly dataset data to an Amazon S3 bucket or an Amazon SFTP folder.

About this task

After you create a data pipeline for export, the pipeline is enabled and file processing starts based on the export schedule. You can disable and enable pipelines from the Data Pipelines grid.
Note: Disabling a pipeline does not disconnect the data source but it prevents data from being exported. Enabling a data pipeline starts file process and resumes the polling of connected data sources
You can delete a data pipeline by clicking Actions and selecting Delete Pipeline.
Note: You cannot delete a pipeline if it was used to download or transform a file or if it is connected to a data source. Disable the pipeline first.

If you need to perform an immediate export of data from the data pipeline and you do not want to wait for the scheduled export which occurs overnight, you can perform a manual export of the data. Select the data pipeline and click Actions and select Manual Refresh to export the data. The status of the file transfer is available from the Data Export Status tab.

Procedure

  1. As a system administrator, from the main menu, click Admin > Data Flow Automation to open the Data Services grid.
  2. On the Data Pipeline tab, click Add New Pipeline.
  3. Assign a name to the pipeline.
  4. Add a description.
  5. Set the type of pipeline to a Data export.
  6. Set the dataset to Monthly Dataset.
  7. Set the destination as either Amazon S3 or SFTP Data Export Data Service.
  8. Click Save.

Results

The pipeline is created and is ready to process files.