Hi Guys is there any other procedure of implementing multi threading in WTX other than in launcher.
Thanks & Regards,
Pinned topic Implementation of Multithreading in WTX without using launcher
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
Updated on 2012-05-16T16:33:27Z at 2012-05-16T16:33:27Z by FormerBuckeye
LewS 0600013M7A19 Posts
Re: Implementation of Multithreading in WTX without using launcher2012-04-20T14:42:57ZThis is the accepted answer. This is the accepted answer.Hi,
Besides using WTX APIs (C, Java, etc.), WTX for WMB, WTX for WESB and WTX for Sterling Integrator (all under WTX for Integration Servers) can multi-thread WTX maps.
Vinothkumar_Dasarathan 270002DFXH5 Posts
Re: Implementation of Multithreading in WTX without using launcher2012-05-15T19:51:50ZThis is the accepted answer. This is the accepted answer.Hi ,
Could you please explain me , how to multi thread a map , without using the Launcher ...
Thanks in advance .
LewS 0600013M7A19 Posts
Re: Implementation of Multithreading in WTX without using launcher2012-05-16T15:01:34ZThis is the accepted answer. This is the accepted answer.Hi,
Multi-threading a map in WTX outside of the launcher requires the map itself to use non-common files (unique workfiles or workspace in memory, unique or no audit files, unique output files). Basically any file being written to should have a unique name across multiple instances.
Next, it really depends on how the maps are set to run.
In WTX for WMB, for example, there is a setting in the BAR file "Additional Instances" that allow multiple threads of the same map to be active. If the input node to the flow has multiple triggers to trigger the flow, multiple instances will be active at the same time (thus being multi-threaded).
The command server cannot run multi-threaded (although, it can run multiple instances of the same map, which is multi-process). If you run multiple processes of the command server with the same map at the same time, and you do not have unique files being written to across these instances, you will either get mapping errors, or unpredictable results.
FormerBuckeye 270000633127 Posts
Re: Implementation of Multithreading in WTX without using launcher2012-05-16T16:33:27ZThis is the accepted answer. This is the accepted answer.I have a large inquiry file that one trading partner needs split into about 200 small files to process. Another trading partner needs the file split into 4 files. Using the crude, but "Keep It Simple, Stupid" technique:
- Root batch job does full translation of large input file using map1 to verify quality and generate large file for most trading partners.
- Root batch job roughly splits the large input file into two files in the input format using map2.
- Root batch job spawns second batch job that executes on second procesor.
- Root batch job uses map3 to translate 1st half of large file into 2 files and map4 to translate 1st half of large file into 100 files.
- Spawned batch job uses copies of map3 & map4 to do the above to the 2nd half of the large file.
When I multithreaded the job, processing wall time dropped from 4+ hours to under 2 hours.