Topic
  • 5 replies
  • Latest Post - ‏2012-05-16T16:33:27Z by FormerBuckeye
sunil0548
sunil0548
16 Posts

Pinned topic Implementation of Multithreading in WTX without using launcher

‏2012-03-30T10:15:19Z |
Hi Guys is there any other procedure of implementing multi threading in WTX other than in launcher.

Thanks & Regards,
Sunil
Updated on 2012-05-16T16:33:27Z at 2012-05-16T16:33:27Z by FormerBuckeye
  • karthikpc
    karthikpc
    41 Posts

    Re: Implementation of Multithreading in WTX without using launcher

    ‏2012-04-04T15:25:10Z  
    Hi,

    WTX with API will do the same.

    karthik
  • LewS
    LewS
    19 Posts

    Re: Implementation of Multithreading in WTX without using launcher

    ‏2012-04-20T14:42:57Z  
    Hi,

    Besides using WTX APIs (C, Java, etc.), WTX for WMB, WTX for WESB and WTX for Sterling Integrator (all under WTX for Integration Servers) can multi-thread WTX maps.

    -Regares
    -lew
  • Vinothkumar_Dasarathan
    5 Posts

    Re: Implementation of Multithreading in WTX without using launcher

    ‏2012-05-15T19:51:50Z  
    Hi ,

    Could you please explain me , how to multi thread a map , without using the Launcher ...

    Thanks in advance .

    Vinoth.
  • LewS
    LewS
    19 Posts

    Re: Implementation of Multithreading in WTX without using launcher

    ‏2012-05-16T15:01:34Z  
    Hi,

    Multi-threading a map in WTX outside of the launcher requires the map itself to use non-common files (unique workfiles or workspace in memory, unique or no audit files, unique output files). Basically any file being written to should have a unique name across multiple instances.

    Next, it really depends on how the maps are set to run.

    In WTX for WMB, for example, there is a setting in the BAR file "Additional Instances" that allow multiple threads of the same map to be active. If the input node to the flow has multiple triggers to trigger the flow, multiple instances will be active at the same time (thus being multi-threaded).

    The command server cannot run multi-threaded (although, it can run multiple instances of the same map, which is multi-process). If you run multiple processes of the command server with the same map at the same time, and you do not have unique files being written to across these instances, you will either get mapping errors, or unpredictable results.

    -regards
    -lew
  • FormerBuckeye
    FormerBuckeye
    27 Posts

    Re: Implementation of Multithreading in WTX without using launcher

    ‏2012-05-16T16:33:27Z  
    I have a large inquiry file that one trading partner needs split into about 200 small files to process. Another trading partner needs the file split into 4 files. Using the crude, but "Keep It Simple, Stupid" technique:

    • Root batch job does full translation of large input file using map1 to verify quality and generate large file for most trading partners.

    • Root batch job roughly splits the large input file into two files in the input format using map2.

    • Root batch job spawns second batch job that executes on second procesor.

    • Root batch job uses map3 to translate 1st half of large file into 2 files and map4 to translate 1st half of large file into 100 files.

    • Spawned batch job uses copies of map3 & map4 to do the above to the 2nd half of the large file.

    When I multithreaded the job, processing wall time dropped from 4+ hours to under 2 hours.