Pinned topic WTX Crashing when processing large numbers of files
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
We have an issue where WTX is crashing when large numbers of files are received at once. The service has to be restarted and the files fed back through at a lesser rate. Does anyone have any suggestions as to what can be done to alleviate this? Thanks in advance.
Updated on 2013-01-24T21:46:13Z at 2013-01-24T21:46:13Z by brian1305
Re: WTX Crashing when processing large numbers of files2013-01-03T16:28:20ZThis is the accepted answer. This is the accepted answer.What do you mean WTX is "crashing"? Be specific on what is happening befor the "crash" occurs. How are the files "received"? What is the maximum size of a file? How are you executing a single map or one map per file? To which WTX "service" are you referring? What release of WTX? On what OS?
Re: WTX Crashing when processing large numbers of files2013-01-03T21:21:30ZThis is the accepted answer. This is the accepted answer.
- SystemAdmin 110000D4XK
Re: WTX Crashing when processing large numbers of files2013-01-04T15:51:22ZThis is the accepted answer. This is the accepted answer.
- brian1305 27000447U1
To single thread the map, that is, process one file at a time, (which I assume you don't want to do and I don't think will work with the number of files you have):
Option 1: In Map Settings, set MapAudit->AuditLocation=File->FileName=Custom->Action=Create->Value=some file name
Option 2: In Map Settings, set WorkSpace->WorkFilePrefix=MapName
Either of these will force the map to write to a single file so subsequent maps will go into Retry mode when they find the map files "locked". You could leave the above settings at default settings and set Map Settings: Retry->Swithch=ON, MaxAttempts=some number but with thousands of input files, this isn't practical. You'd end up with maps timing out.
To multi-thread the map:
Check Help screens for:
"Running multiple concurrent instances of maps"
In DTX.ini are a number of parameters that relate to maximum number of map instances. Unfortunately, I can't find the documentation in the Help screens. Some of these settings can be overwritten from within the Integration Flow Designer.
Check the Performance Recommendations section of the Help screens.
You have the option to design multiple map instances in the IFD that all watch the same directory but I don't believe this would help your situation. Multiple instances of the Launcher on multiple systems would help but multiple licenses would be required.
Unfortunately I can't come up with any specific answers for you even though I know I have solved similar issues in the past. Hopefully it will come to me or someone will jump in with more specific suggestions.