Restart capability

You can set certain stage properties to run the FTP operation in restartable mode.

You can specify that the FTP operation runs in restartable mode. To do this you set the stage properties as follows:

1. Specify a restartable mode of restartable transfer.
2. Specify a unique job id for the transfer
3. Optionally specify a checkpoint directory for the transfer directory (if you do not specify a checkpoint directory, the current working directory is used)

When you run the job that performs the FTP operation, information about the transfer is written to a restart directory identified by the job id located in the checkpoint directory prefixed with the string "pftp_jobid_". For example, if you specify a job_id of 100 and a checkpoint directory of /home/bgamsworth/checkpoint the files would be written to /home/bgamsworth/checkpoint/pftp_jobid_100.

If the FTP operation does not succeed, you can rerun the same job with the restartable mode set to restart transfer or abandon transfer. For a production environment you could build a job sequence that performed the transfer, then tested whether it was successful. If it was not another job in the sequence could use an FTP stage with the restart transfer option to attempt the transfer again using the information in the restart directory. .

For get operations, InfoSphere® DataStage® reinitiates the FTP transfer at the file boundary. The transfer of the files that failed half way is restarted from the beginning or zero file location. The file URIs that were transferred completely are not transferred again. Subsequently, the downloaded URIs are imported to the dataset from the temporary folder path.

If the operation repeatedly fails, you can use the abandon transfer option to abandon the transfer and clear the temporary restart directory.