This topic has been locked.
5 replies Latest Post - 2013-01-24T21:46:13Z by brian1305
Pinned topic WTX Crashing when processing large numbers of files
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
We have an issue where WTX is crashing when large numbers of files are received at once. The service has to be restarted and the files fed back through at a lesser rate. Does anyone have any suggestions as to what can be done to alleviate this? Thanks in advance.
Updated on 2013-01-24T21:46:13Z at 2013-01-24T21:46:13Z by brian1305
Re: WTX Crashing when processing large numbers of files2013-01-03T16:28:20Z in response to brian1305What do you mean WTX is "crashing"? Be specific on what is happening befor the "crash" occurs. How are the files "received"? What is the maximum size of a file? How are you executing a single map or one map per file? To which WTX "service" are you referring? What release of WTX? On what OS?
Re: WTX Crashing when processing large numbers of files2013-01-03T21:21:30Z in response to SystemAdminThe files are brought in via FTP into a folder, thousands at a time. They are transferred to another folder, where WTX starts processing them via our 'move_inbound' msl service. Each file is a single transaction, somewhere in the 1K range, size-wise. These files are all processed via a single map. The service is called 'dtx81_launcher', which points to eventserverservice.exe in the WTX directory. We are running version 8.1 on Windows 2003.
Re: WTX Crashing when processing large numbers of files2013-01-04T15:51:22Z in response to brian1305OK. That helps. It sounds like Launcher is firing off too many instances of the map. I just realized I haven't worked with Launcher for 4 years but hopefully I recall enough. Also, let's hope someone else jumps in to help.
To single thread the map, that is, process one file at a time, (which I assume you don't want to do and I don't think will work with the number of files you have):
Option 1: In Map Settings, set MapAudit->AuditLocation=File->FileName=Custom->Action=Create->Value=some file name
Option 2: In Map Settings, set WorkSpace->WorkFilePrefix=MapName
Either of these will force the map to write to a single file so subsequent maps will go into Retry mode when they find the map files "locked". You could leave the above settings at default settings and set Map Settings: Retry->Swithch=ON, MaxAttempts=some number but with thousands of input files, this isn't practical. You'd end up with maps timing out.
To multi-thread the map:
Check Help screens for:
"Running multiple concurrent instances of maps"
In DTX.ini are a number of parameters that relate to maximum number of map instances. Unfortunately, I can't find the documentation in the Help screens. Some of these settings can be overwritten from within the Integration Flow Designer.
Check the Performance Recommendations section of the Help screens.
You have the option to design multiple map instances in the IFD that all watch the same directory but I don't believe this would help your situation. Multiple instances of the Launcher on multiple systems would help but multiple licenses would be required.
Unfortunately I can't come up with any specific answers for you even though I know I have solved similar issues in the past. Hopefully it will come to me or someone will jump in with more specific suggestions.