In this Blog am writing details about few important performance parameters that helps with tuning translation service.
Many a times the users are reporting issues with translator performance and ways to resolve the issues. We usually see users reporting outofmemory issues with translation due to huge size files, files that contains many loops, sql maps that does select huge number of records, output data gets truncated during processing etc. The performance may be depends on physical memory available on the server and not enough CPU cores available for processing etc... However as a first step the user can look the problem and see if tuning the key translator parameters are helps to resolve the issue. To translate very large files or to process files without any issues, it may be necessary to alter values of the below parameters to improve performance or to resolve issues.
Few key translation parameters that the user want to alter. These parameters can be found at translator.properties_b2b_base_ext (at <SBI_INSTALL_DIR>/properties directory)
storage.KeyBatchSize
storage.DataBatchSize
storage.maxKeyFileSize
storage.LargeFileThreshold
storage.CacheSize
storage.SwapSize
1. storage.KeyBatchSize - This is the Size of the key record write batch. When storage keys (indices) need paged to disk, the translator pages keys in batches specified by storage.KeyBatchSize. Valid value is any integer. Default is 1000.
Example: storage.KeyBatchSize=1000
2. storage.DataBatchSize
The Size of data record write batch. (i.e) The size of the data record write batch when storage data (field data) needs paged to disk. The translator pages data in batches specified by storage.DataBatchSize. Valid value is any integer. Default is 1000.
Example: storage.DataBatchSize=1000
3. storage.maxKeyFileSize
This Determines max initial offset into the key file. To translate very large files, it may be necessary to increase value to improve performance.
Example: storage.maxKeyFileSize=1000000
4. storage.LargeFileThreshold
This is the size of input stream that triggers large file support.
Example: storage.LargeFileThreshold=15000000
When the translator is in LargeFileThreshold mode the data is stored on disk, not in memory. So when the translator is processing a data file and is in LargeFileThreshold mode, the translator will store the segment read from the input file in memory for the time it takes to process that segment in the map. When it reads the next segment from the input file, it stores the previous segment on disk and places the new segment in memory. This means that only the current segment is in memory at any given point rather than all the segments read up to that point.
Please refer the below link for more details about this property:
http://www-01.ibm.com/support/docview.wss?uid=swg21517418
http://www-01.ibm.com/support/docview.wss?uid=swg21544507
5. storage.CacheSize
The Size of the storage cache. For each storage group, this is the number of instances of each group that will be kept in memory. Valid value is any
integer. Default is 7000.
Example: storage.CacheSize=7000
6. storage.SwapSize
The Number of entries in the storage cache to swap. For each storage group, this is the number of instances to page to disk when the number
of instances equals the value of storage.CacheSize. Valid value is any integer. Default is 6000.
Example: storage.SwapSize=6000
Refer the below link for more details about a scenario:
http://www-01.ibm.com/support/docview.wss?uid=swg21554612
If you would like to override these parameters with required values you can override using customer_overrides.properties file as below.
Example:
translator.storage.maxKeyFileSize=10000000
translator.storage.CacheSize=80000
translator.storage.SwapSize=70000
translator.storage.KeyBatchSize=5000
translator.storage.DataBatchSize=5000
translator.storage.LargeFileThreshold=500000000
If there are any queries/comments please post your comments and we will look into that accordingly.
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS3JSW","label":"IBM Sterling B2B Integrator"},"Component":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"","Edition":"","Line of Business":{"code":"LOB59","label":"Sustainability Software"}}]