Scheduling batch uploads

After the Header File is created, populated with record information, and saved as tab-delimited, you are ready to automate the upload process and schedule batch uploads with Data Integrator.

About this task

When a record is first created, it is in a special state called null. Records in the null state disappear after the operation that currently uses the record is done. It is important for the Data Integrator to run an action on newly created records to take them out of the null state. If the action does not take the record out of the null state, the record disappears immediately after it is created. All of the options in the Action list are actions that transition records out of the null state.

Often it turns out that the interval you specify between times that the Data Integrator runs divides evenly into 24 hours. If so, data is read from the file at the same times every day.

If you choose an interval that does not divide evenly into 24 hours, such as 5 hours, data is read from the file at the same times every day. In other words, every day, at the time that is specified in the Reset clock every day at field, data is scheduled to be read from the specified file. It is then rescheduled at the specified interval. For example, assume that the upload process begins every day at 9:00 AM. If you set the interval for 5 hours, 0 minutes, the batch process would run starting at 9:00 AM and increment every 5 hours for the next 24 hours. Therefore, it would run every day at 9:00 AM, 2:00 PM, 7:00 PM, 12:00 AM, 5:00 AM. It would then start again at 9:00 AM, iterating through the same time increments again.

A running agent runs about every 10 seconds. The agent iterates through all the batch process objects that are created and based on the current time, runs the batch objects at the appropriate time. Since the exact time when the agent runs is unknown, the IBM® TRIRIGA® Application Platform puts a 1-minute buffer on the start time of each time increment. In the example, if the agent runs at 9:00:10, the process runs since the platform checks between 9:00 AM and 9:01 AM. In the example with a 5-hour increment, the batch files are run if the current time is between 9:00 AM - 9:01 AM, 2:00 PM - 2:01 PM, 7:00 PM - 7:01 PM, 12:00 AM - 12:01 AM, and 5:00 AM - 5:01 AM.

Procedure

  1. From the main navigation bar, select Tools > Administration > Data Integrator.
  2. Verify that the values in the Module, Business Object, and Form fields match the Header File. The value of Import Type must be Add.
    The Add value means that if a row contains values for a record’s Publish Name fields that match an existing record, the other values in the row update the record. If a row contains values for a record’s Publish Name fields that do not match an existing record, the data in the row is used to create a new record.
  3. From the Action list, select the action to run on a record after the Data Integrator creates or updates it.
    This list contains the valid actions for a new record, based on the state transition family that is used by the selected business object.
  4. If a workflow must start after the Data Integrator creates a record, you must start the workflow with a sub action that is attached to the action in the Action list.
  5. The only value of File Type is tab delimited (*.txt). But from the File Char Set list, select the character set in which the imported file was written.
  6. Select the Batch Upload check box. The fields that are labeled Upload Batch Data Every ___ Hours ___ Minutes control how often the Data Integrator is scheduled to read data.
  7. In the Upload Batch Data Every field, set the time increment for the upload intervals that start from the time in the Reset clock every day at field. The default is 0 hours and 0 minutes.
  8. Set Reset clock every day at to the time you want the batch upload process to start each day.
    If Reset clock every day at contains a value greater than 24 hours, the upload occurs only one time per day at the designated Reset clock every day at time.
  9. Specify the Header File name in the Filename (*.txt) field. The file must be in a directory on which the IBM TRIRIGA Application Platform runs.
    If the designated file is found in the appropriate location, the platform processes the file and places the process time in the Last Run Time field of the Data Upload business object.
  10. Enter the Confirmation eMail address.
    After each attempt that the Data Integrator makes to batch upload a file, it sends an email to the address specified. This email is sent whether or not the upload is successful.
  11. Select Upload File.
  12. Select OK.
    The platform schedules the upload of the specified file, and continues to schedule the file upload at the specified interval indefinitely. The data is read from the specified file. After each upload process is complete, an email notification is sent to the Confirmation eMail address.
  13. Verify that all records were imported correctly by viewing the related portal or page.