WebSphere Transformation Extender performance improvement guide

Resolving The Problem

A WTX map operates essentially like a program. A map is constrained by the speed of its slowest element. It can be disk I/O-bound, communications I/O-bound, or CPU-bound. Even if a map is doing very little actual transformation work, it can still be CPU bound due to the amount of CPU required to extract data from a source (database, SAP etc.).

1. Raw CPU speed. The faster the CPU the better. Performance improves roughly linearly with CPU speed.
2. Multiple CPUs. If the maps to be run are multi-threaded or multiple different maps are to be run, then adding additional CPUs will help because each mapping thread runs on one CPU only. If the fundamental job is processing one single map, or a map that calls RUN maps, then extra CPUs will not help. A map will attempt to consume as much CPU resources as it needs. It can vary from a small amount up to 100% of the CPU cycles depending on the data complexity, mapping rules and functions, map complexity and other factors.
3. Main memory. The more the better. If you will be using large in-memory work area or memory based adapters, add memory to accommodate these files and data to avoid paging overhead.
4. Disk I/O. Raw disk I/O speed will affect performance for local sources and/or targets. I/O performance to databases is often CPU intensive and performance will be aided by a faster CPU if the database is local to WTX.
5. Communications (network) I/O. Remote data sources and data targets will slow processing time down. LANs will usually be slower than a good local hard disk, disk array, or SAN.
6. Adapters. If using ODBC to access remote data sources or targets, the best ODBC drivers will support features like array binding and use internal pipelining to provide good performance. Other adapters may be able to be tuned to improve performance. Contact the adapter's vendor for information.
7. Other programs. Applications running concurrently on the WTX server machine will slow down the performance of WTX maps and systems.
8. Hyper-threading. Hyperthreaded systems are not as fast as actual CPUs or core. Different hardware will have difference specifications. Check with the vendor for performance ratios.

• Upgrade to faster CPU(s) and add more cache.
• Add or allocate more main memory.
• Move source and target data files or tables to separate local disks and enable dual asynchronous I/O paths (e.g., dual SCSI controllers).
• Add additional CPUs.
• Use the fastest driver available for a data source or target. Often native database adapters are faster than ODBC.

• Follow map tuning tips in the documentation and use the map Profiler.
• See How to save memory and improve performance in WebSphere Transformation Extender maps and systems.
• Use stored procedures rather than local SQL in map rules where possible.

• Install additional run time environments. If you are reaching a defined user limit that can not be raised, or the 32 bit program data size limitation, or other user or system specific limit, another installation of the run time engine, under a different user ID and path can be used to overcome these limitations. This applies to both the Command Server, DK and Launcher.
• Do not run the Command Server, Development Kit and Launcher in the same environment or installation. This can cause resource and memory conflicts.
• Check the compoundsystem.txt or msl_name-timestamp.txt file for conflicts in output cards and resources. To produce this file, enable the LaucherLog setting in the configuration file. The file will be created in the logs directory.
• Check that the map and adapter tracing is turned off in the *.txt files generated in previous item. To turn off all tracing, enable the MapAdapterTraceOff setting in the configuration file. Check the map rules for RUN map calls and the RUN maps themselves for tracing settings.
• Check to see if work files for look ups can be reused. If the lookup files are static, then the work file can be reused. If they change often, using a small database table may help.
• Check for UNC paths where mapped or mounted drives could be used. UNC paths add network overhead.
• Use local drives rather than drives across a network.
• Check for non-unique audit logs. The logs may not be useful and can be turned off to save I/O. If the non-unique audit log was the only cause for a map to be single threaded, this will allow multi-threading.
• If the result of tuning still does not produce a system with adequate performance or the systems are consuming more resources than are available on the server, you may consider a larger server or setting up a load balancing system with more than one server.

• Different operating systems (OS) can have kernel and other parameters that can impact map or system performance. Consult with your OS Administrator for OS specific tuning.
• Keep up with the vendor’s recommendations for OS and driver patches and updates.
• On Windows®, set the virtual memory (page file) to be a fixed size rather than let the OS manage the file.
• On Windows, defragment the drives regularly.