Troubleshooting
Problem
FSA Collect waits until fileModThreshold is complete
Symptom
Enhance the fileModTimeThreshold functionality of File System Adapter.
Current functionality:
Sets the file modification time threshold (in seconds) for files to be collected. A file is collected only if the modification time of the file is older than the number of seconds specified. This prevents premature collection of a file. Defaults to 30 seconds if you do not specify a value.
If the business process has File System Adapter with fileModTimeThreshold value of 900 (15 mins) and the File System Adapter does collection. File System Adapter will collect the file immediately as long as the file at least 15 mins old. If the File is less than 15 mins old, the File System Adapter waits until file gets 15 mins old and then collect the file. In short, while the File System Adapter is waiting to get the file aged to what mentioned in fileModTimeThreshold, the business process shows the state of Active in BP Monitor.
The situation gets even worse, if the directory where the File System Adapter collecting the files from is written with files consistently, then File System Adapter waits for each file to get 15 mins old, collect it and then move to the next file.
Consider a scenario where File System Adapter is collecting file 1 which is not 15 mins old. The FSA will wait on a file 1 to get 15 mins old. While the FSA is waiting, 2 more files (file 2, file 3) are written to the directory where FSA is collecting from. As soon as file 1 becomes 15 mins old, it is collected, but then FSA waits until file 2 gets 15 mins old then collect file 2. Then FSA wait until file 3 gets 15 mins old, then collect the file. Meanwhile if there are more files written to the directory, the waiting continues.
Enhance the functionality such that the FSA should not collect the file if the file is not 15 mins old and move to the next file. FSA should not wait until the file gets old enough to collect.
Enhanced functionality as follows:
A new Parameter thresholdWait added to the FSA configuration. When setting the thresholdWait from GPM, it can be set to Yes/No from dropdown.
The File System Adapter functionality is enhanced as follows:
If collectMultiple = Yes and bootstrap = No then
If FSA has fileModTimeThreshold specified then
If thresholdWait = Yes then (Note: this is equivalent to current processing)
FSA collect the file immediately if file is older than fileModTimeThreshold.
FSA wait if the file to be collected is not old enough. When the File gets old enough, it collect it and then move to the next file to collect.
While the FSA is waiting for file to collect, the business process stays in Active State.
FSA works on the list of files. Once the FSA listed the files and started collecting, if any new files comes in the collection directory, the new files will not be collected in that business process Run.
If thresholdWait = No then
FSA collect the file immediately if file is older than fileModTimeThreshold.
FSA does not wait if the file to be collected is not old enough. Instead it skip the file and proceed to collect next file from the list.
The status report on FSA step in the business process execution will state that the file not collected as it's not old enough.
FSA works on the list of files. Once the FSA listed the files and started collecting, if any new files comes in the collection directory, the new files will not be collected in that business process Run.
If bootstrap = Yes then (Note: FSA processes the same whether collectMultiple is yes or no when bootstrap=yes)
If FSA has fileModTimeThreshold specified then (Note: it defaults to 60)
All the files in the collection directory get collected if older than the file modification timestamp.
If FSA has fileModTimeThreshold set to zero then
All the files in the collection directory gets collected irrespective of the file modification timestamp.
Error Message
No error message.
Log InLog in to view more of this document
Historical Number
NFX1098
Was this topic helpful?
Document Information
Modified date:
10 January 2020
UID
swg21555707