Question & Answer
Question
Attempts to do Gateway data transfer from remote server using FTP protocol takes a long time to complete.
Cause
One of the problems with FTP transfer is with the timestamp file, where it does not exists when Gateway starts for the first time.
Without any timestamp to guide the Gateway, it would transfer all the data which fits the criteria set in INPUT_FILE_DESCRIPTION from the location provided in REMOTE_DIR
which are configured in the TransferConfig.pm file.
The same problem would happen if timestamp file stores old date. This will cause too many files to be transferred. If the remote directory stores archive data that fits the same criteria, then the initial extraction/transfer could be time consuming because the Gateway would extract the archive data as well.
Answer
To overcome this problem, locate the TransferConfig.pm in Gateway Configuration folder, and look for the TIMESTAMP_FILE . This can be an absolute path, or a path relative to the release folder.
A sample FTP Timestamp file is given below :
In order to transfer the more current files only :
i) Edit (vi) the timestamp file above.
ii) The value of LAST_TIME needs to be changed to most current timestamp. Choose a date/time required for the oldest file to be transferred.
iii) This value is unix (epoch) value, an example of obtaining this is give below :
virtuo@server: echo " Date `date` is ` perl -e 'print time(), "\n" ' `" Date Tue Dec 27 12:42:52 MYT 2011 is 1324960972 |
iv) You would change the
LAST_TIMEvariable to 1324960972 if you wish to only obtain files dated after "Tue Dec 27 12:42:52 MYT 2011"
Was this topic helpful?
Document Information
Modified date:
21 June 2018
UID
swg21575999