Example–Repeating a loop for each file in a directory

The data flow in this example reads a source file, transforms the data, and writes the data to a target database. You want this one data flow to run a number of times in succession, each time with a different source file. One way to do this is to design one data flow for each source file that you want to read. A more efficient way to do this is to design a control flow that uses an iterator.

If the data flow fails, you want an e-mail sent to an operator.

To define an iterator operator for the files in a directory:

1. In your data flow, define File Name as a variable for your file import.
   Tip: You define this variable when you create the data flow in the data flow editor.
2. In your control flow, place an iterator operator in the canvas.
3. Define the properties for the iterator.
   1. Select the following type of iteration: Repeat loop for each file in a directory.
   2. Select the directory that contains the source files.
   3. Click to select the File Name variable.
4. Place a data flow operator in the canvas and define the data flow to be the one that you created in the Design Studio.
5. Connect the main processing loop to the iterator and end iterator operators.
   1. Connect the loop port of the iterator operator to the input port of the data flow operator.
      Figure 1. Loop port of an iterator operator connected to the input port of a data flow operator

      
   2. Connect the On-success port of the data flow operator to the input port of the end iterator.
      Figure 2. On-success port of a data flow operator connected to the input port of an end iterator.

      
6. Define the error processing loop.
   1. Place the e-mail operator in the canvas and define its properties.
   2. Connect the on-failure port of the data flow operator to the input port for the e-mail operator.
      Figure 3. On-failure port of a data flow operator connected to the input port for an e-mail operator

      

When the control flow is run, the data flow is run one time for each source file.