Product Documentation
Abstract
The WebSphere Data Interchange implementation of Management Reporting components are being modified as a result of several customer reported issues found with the Management Reporting processing involving Data Transformation and minor enhancements for Send/Receive Translation Reporting for the KNOWN trading partner . These modifications have been made under APAR PI13317, which is included with PTF UI18489 for WDI V3.3.0 for z/OS or with Server Fix Pack 10 for WDI V3.3.0 MultiPlatform for AIX and Windows.
The reports effected are PERFORM TRADING PARTNER CAPABILITY DATA EXTRACT and PERFORM TRANSACTION ACTIVITY DATA EXTRACT. Analysis finds that corrections in processing for the issues related to these two reports will cause a change in behavior for any customers using these two Management Reporting features.
During the analysis, several other design related issues were found and corrected. In an attempt to minimize these changes in behavior, the WDI development team has introduced an option for Management Reporting execution which will produce both the old results and new results. Some effects of applying APAR PI13317 cannot be avoided.
Most effects have been isolated using a new reporting option described in this document.
Applying APAR PI13317, will result in many more rows inserted into the Management Reporting pending and statistics data base tables ONLY if the WDI Management Reporting feature is active. The amount of space may triple the space currently required. Before APAR PI13317 is applied, Data Transformation currently inserts 1 row for each XML and ADF de-envelope process. After APAR PI13317 is applied, Data Transformation will insert 1 row for each EDI, XML, and ADF de-envelope process with an additional 2 rows for each document being transformed. These will be inserted into the EDIMRPR and EDIMRPS pending tables. The statistics tables EDIMRRT and EDIMRST will be updated based on the entries in the EDIMRPR an EDIMRPS pending tables.
These tables may need more table space. Note that customer using only Send/Receive maps are not affected by the additional table space that may be needed.
Tables affected: EDIMRPR, EDIMRPS, EDIMRRT, and EDIMRST.
Appendix C in the WDI 3.3 Programers Reference Guide contains sample space calculations for all DB2 table spaces. The size of the records has not changed, the number of rows inserted with each Data Transformation de-envelope process has changed.
To determine if the Management Reporting feature is currently active, customers must view the Application Defaults Profiles under the Environment section using WDI Client.
WDI Client->Environment->Application Defaults
Select the Application Defaults profile being used for Translation/Transformation. The default profile name is "EDIFFS". If Management Reporting is active, the box to the left of Management Reporting will contain a check mark. Although the "EDIFFS" Application Defaults is most commonly used, customers may customize the profiles that are used for processing options. Other profile members may exist and should be identified for active Management Reporting.
Content
The WDI implementation for Management reporting was originally designed for reporting on the EDI process itself. It was primarily concerned with questions like "How many purchase orders did I send to each of my trading partners last month?" or "Which trading partners am I doing purchase orders with at the X12 V2R3 level?" These reports provide insight into exchange activity with Trading Partners showing the partner's ability to process specific documents and their history of doing so in test or production. To generate the output records, Management Reporting gathers information from many different database tables within the WDI product in addition to the Management Reporting statistics tables.
Customer Reported Issues
Management Reporting Trading Partner Capability and Activity Data Extract reports for Data Transformations are in error and missing data. After running PERFORM UPDATE STATISTICS, management reporting reports for Data Transformation processing are incorrect.
Examples:
- 1. PERFORM TRADING PARTNER CAPABILITY DATA EXTRACT WHERE STDTRID(810)
The EDIQUERY file contains hundreds records all with *NODATE*
and 0 count for “Total number of transactions 354-368” and
“Total errors 369-383” and *NODATE* for “Measurement date
342-349”. The report (Transaction ID 318-325) also shows more
rows than just for the STDTRID as specified on the WHERE clause.
2. PERFORM TRANSACTION ACTIVITY DATA EXTRACT WHERE MAPID(8554010)
The EDIQUERY file correctly contains only rows for this map,
but the counts are all 0 with *NODATE*
It was observed that EDI to ADF (direction R) does not save any info (EDIMRPR) during transformation, so it would have zero counts (and no date) as a net result in the report. Whereas ADF to EDI does save to EDIMRPS, hence EDIMRST is updated with counts during update statistics.
Problem Conclusion
Send/Receive Translation is very clear on direction, identification of the source and target documents, and what is a Send or Receive Management Reporting pending record. The idea of Send and Receive should be uniquely applied to those map types in which EDI is uniquely the input or output. This processing will not change.
Unlike the Send/Receive translation, Data Transformation processing cannot be clearly identified as a Send or Receive and statistics should be reported for both Trading Partners involved rather than the single "external" Trading Partner. This maintains the description of these reports answering the question "what can this Trading Partner do and what have they done already?"
With the corrections for the customer reported issues along with the additional issues found during analysis, the impact to existing implementations using Management Reporting would have been extensive and required new optional processing to enable the corrections. However, some changes in behavior of existing features was unavoidable. Changes in behavior for ALL customers applying APAR PI13317 are listed below:
EDI Standard source document processing has been corrected to insert a Management Reporting measurement into one of the pending tables. Current processing determines the Receive or Send Pending table based on Receiving and Sending Trading Partner types.
Data Transformation new processing will insert 1 additional Management Reporting measurement into both of the pending tables with a new measurement ID “xxDT” for Data Transformation. The Source (Input) document information will insert into the Management Reporting Receive pending table based on the Sending Trading Partner nickname and the Target (Output) document information will insert into the Management Reporting Send pending table based on the Receiving Trading Partner nickname. This modification is needed to align the Data Transformation processing flow to the Receive/Send processing flow.
The Management Reporting Trading Partner Capability statistics update for accumulating values has been corrected for the old and new reporting options.
Measurements for Data Transformation EDI source document information will now be included. This is the only correction to the current reporting results. Other source document information reported will remain unchanged.
If the new optional processing is NOT USED:
Any corrections with selection criteria or incorrect and incomplete information will not be addressed in the old reporting results for example PERFORM TRADING PARTNER CAPABILITY DATA EXTRACT WHERE STDTRID(810) will continue to show more rows than just for the STDTRID as specified on the WHERE clause. This issue has been corrected with the new reporting option (see Implementation section of this document).
Any future corrections will ONLY be addressed with the new optional processing.
Incorrect Behaviors
The current processing for Data Transformation EDI input (Receive) is not working because no records are being inserted into the pending Management Reporting tables.
The Data Transformation outbound (Send) Management Reporting output records are incomplete, missing information from other tables, null padding, and does not match the selection criteria. The only selection criterion that is working with Data Transformation measurements is TPNICKN and MAPID.
There are no fields in the Management Reporting output records for XML document identification. Receive Management Reporting pending records have the EDI document (length 8), Send Management Reporting pending records have the Data Format document (length 16) and there is no field for XML document (length 30).
Data Transformation does not always fit the Send/Receive model and the current processing looks at the Receiving and the Sending Trading Partner type to determine if the source document is a Send Management Reporting pending record or a Receive Management Reporting pending record. The direction (Receive/Send) is confusing because the trading partner type is currently being used to identify the direction.
For example, X12TOXML mapping execution produced a Send MR pending record because the Receiving Trading Partner is type ‘E’. The logic below is currently used in the DT De-enveloping process.
If the Receiving trading partner is type ‘A’, then DT generates a Receive MR record.
If the Receiving trading partner is type ‘E’, then DT generates a Send MR record.
If the Receiving trading partner is type ‘B’, then
- If the Sending trading partner is type ‘A’, then DT generates a Send MR record.
If the Sending trading partner is type ‘E’, then DT generates a Receive MR record.
If the Sending trading partner is type ‘B’, then
- If the source document type is ‘E’, then DT generates a Receive MR record.
- Otherwise DT generates a Send MR record.
Management Reporting statistics are gathered during the translation and transformation process. The Sending and Receiving (or application) trading partners may or may not be identified in the Trading Partner Profiles. Management Reporting Statistics are updated based on the Trading Partner Profiles found.
- If the trading partner is not found in the Trading Partner Profile, the "ANY" or "UNKNOWN" trading partners will be used to select the specific Usage or Rule which identifies the mapping and processing options to be used. The Management Reporting statistics are updated based on the "UNKNOWN" trading partner. If there are no usages or rules selection for the "UNKNOWN" trading partner, Management Reporting has no access to these records.
If the trading partner is found in the Trading Partner profile , the "ANY", "KNOWN", or Trading Partner Profile will be used to select the specific Usage or Rule which identifies the mapping and processing options to be used. If the Usage or Rule contains "KNOWN" trading partners, the trading partner statistics are updated using the "real" Trading Partner Profiles found. If there are no usages or rules selection for the "KNOWN" trading partner and Management Reporting has no access to these records.
This "ANY" and "KNOWN" trading partner issue effects both Data Transformation and Send/Receive Management Reporting Queries.
The Management Reporting Trading Partner Capability statistics are not accumulating. These pending records are being applied the same as the daily statistics. This issue affects old reporting for both Data Transformation and Send/Receive.
New Data Transformation Processing
The new Management Reporting option, using the DIR() keyword values for implementation, will show the Data Transformation flow with comparable Send/Receive information. It also enhances the Management Reporting output records to identify Data Transformation vs Send/Receive records and improves performance.
For each Data Transformation de-envelope, the source document and Sending Trading Partner information will generate a Receive Management Reporting pending record. The target document and Receiving Trading Partner information will generate a Send Management Reporting pending record.
To track "pending" activity during Data Transformation, each syntax type will insert a row into the Management Reporting Receiving pending table (EDIPRPR) and Management Reporting Sending pending table (EDIMRPS). The processing will track the TPTID (map id), TPNICKN (the originator or sender), APPLTPID (the target or receiver), INTPID (if identified) that drove transformation to the Rule, DATE, MEASID using "C" for cumulative or "D" for detail rows, then P/T for usage/test indicator, and the transaction counts. Position 3 and 4 of MEASID will be used to indicate this was a Transform map “DT” or Send/Receive map "TR". For Example, DTDT (Detail, Test, Data Transformation) and DPDT (Detail, Production, Data Transformation), CTDT (Cumulative, Test, Data Transformation) and CPTR (Cumulative, Production, Translator).
To minimize impact to current customers using Management Reporting, a pending record will be inserted using the old Data Transformation format (as it is using the Trading Partner type to determine direction) and the new Data Transformation format (working similar to Send/Receive) using sending and receiving (application) Trading Partner nickname values.
The Management Reporting Trading Partner Capability statistics update for accumulating values has been corrected for the old and new reporting options. During the Update Statistics process when applying the pending records to the cumulative records, WDI processing will gather the measurements from the duplicate cumulative records to generate a grand total and remove the duplicate records. This processing will only affect the Management Reporting records for rules and usages being used for transformation and translation that generate pending records. Any unused rules and usages that also have measurements will not be updated or removed.
Implementation
To enable the new reporting option the DIR() keyword is required for the 2 affected reports, PERFORM TRADING PARTNER CAPABILITY DATA EXTRACT and PERFORM TRANSACTION ACTIVITY DATA EXTRACT. The new values for DIR() keyword will only work with these two commands.
To use the new redesigned reporting, customers will be required to use the DIR() keyword.
Value ‘A’ will produce receive/send/inbound DT/outbound DT.
Value ‘I’ will produce receive/inbound DT.
Value ‘O’ will produce send/outbound DT.
If the DIR() keyword is currently being used and customers want to use the new reporting, they must replace the current values “R” and/or “S” with the values “I” and/or “O”. Otherwise the old reporting will be executed. This applies to both DT and S/R processing. Use of these new values will report applicable information in the expanded Management Reporting records for Send/Receive reports.
If the DIR() keyword is currently NOT being used and customers want to use the new reporting, they must add the DIR() keyword to the two reports effected. For example: DIR(A) to select both inbound and outbound, DIR(I) for inbound only, or DIR(O) for outbound only. The two reports effected are PERFORM TRADING PARTNER CAPABILITY DATA EXTRACT and PERFORM TRANSACTION ACTIVITY DATA EXTRACT. Otherwise the old reporting will be executed.
Management Reporting Output
To generate the output records, Data Transformation maps must look at the Rule to find the Source Syntax, Dictionary and Document and the Map Syntax for the Target Syntax, Dictionary and Document. These values along with the Test/Production/Information indicator, source dictionary and document will be extracted from the Rule and will be placed in the filler (expansion areas) at the end of the TP Capability (filler 625) and Transaction Activity (filler 609) output records. The Measurement Application Routing Type and Measurement Application Routing Partner are also included to identify the specific rule or receive usage that was used during inbound or receive processing. These values are not available in the outbound or send Management Reporting Tables and cannot be reported.
The existing Trading Partner and Application Trading Partner fields will contain the information from the Usage or Rule selection. The Measurement Trading Partner and Measurement Application Trading Partner fields have been added to report the information for the "KNOWN" Usage or Rule selection. These "KNOWN" Management Reporting records will be appended at the end of each reporting section. Reporting sections are: Receive Translation, Send Translation, Inbound Data Transformation, Outbound Data Transformation.
Trading partner capability data extract. Replacement for "Filler" at the end of Table 94 on page 309 of the "Utility Commands and File Formats Reference"
| Label | Position | Length | Type |
| Measurement Trading Partner | 400 - 415 | 16 | Char |
| Measurement Application Trading Partner | 416 - 431 | 16 | Char |
| Measurement Application Routing Type | 432 - 432 | 1 | Char |
| Measurement Application Routing Partner | 433 - 467 | 35 | Char |
| Rule/Usage Test/Production/Info | 468 - 468 | 1 | Char |
| DT Source Syntax | 469 - 471 | 3 | Char |
| DT Source Dictionary Id | 472 - 501 | 30 | Char |
| DT Source Document Id | 502 - 531 | 30 | Char |
| DT Target Syntax | 532 - 534 | 3 | Char |
| DT Target Dictionary Id | 535 - 564 | 30 | Char |
| DT Target Document Id | 565 - 594 | 30 | Char |
| Filler | 595 - 1024 | 430 | Char |
Transaction activity data extract. Replacement for "Filler" at the end of Table 96 on page 311 of the "Utility Commands and File Formats Reference"
| Label | Position | Length | Type |
| Measurement Trading Partner | 416 - 431 | 16 | Char |
| Measurement Application Trading Partner | 432 - 447 | 16 | Char |
| Measurement Application Routing Type | 448 - 448 | 1 | Char |
| Measurement Application Routing Partner | 449 - 483 | 35 | Char |
| Rule/Usage Test/Production/Info | 484 - 484 | 1 | Char |
| DT Source Syntax | 485 - 487 | 3 | Char |
| DT Source Dictionary Id | 488 - 517 | 30 | Char |
| DT Source Document Id | 518 - 547 | 30 | Char |
| DT Target Syntax | 548 - 550 | 3 | Char |
| DT Target Dictionary Id | 551 - 580 | 30 | Char |
| DT Target Document Id | 581 - 610 | 30 | Char |
| Filler | 611 - 1024 | 414 | Char |
When identification is possible for Data Transformation (non-XML), the EDI and Data Format document ids and information will be populated in the areas where they exist today with Send/Receive translation. Any EDI syntax used by the Trading Partner populates the standard version, release, description and TRANID. Likewise, Data Format syntax used by the Trading Partner.
The “Direction” output field in the Trading Partner Capability and Transaction Activity records for Data Transformation will contain "I" (Inbound) indicating that this sending Trading Partner supplied the inbound INFILE and "O" (Outbound) indicating this receiving Trading Partner was the Outbound target.
Limitations
WDI processing allows multiple WHERE clauses to be stacked. For example WHERE DIR(R) WHERE DIR(S). Currently Management Reporting reports give both Send/Receive and Data Transformation with each report request so DIR(R) will give output records with both Receive and Inbound Data Transformation. Customers using multiple WHERE DIR() cannot mix the old and new reporting for example WHERE DIR(A) WHERE DIR(S). The first DIR value will be used to determine the new or old reporting.
Duplicate Management Reporting Records
The Management Reporting Measurement ID field position two identifies test or production processing. It is possible with inbound EDI documents to have a measurement id indicating production, for example CPDT, but a test usage or rule selection was used during the processing. When both a test and production usage or rule exist with the exact same trading partner information is defined on a map, this will cause duplicate reporting records for the trading partner.
Both Send/Receive and Data Transformation maps defining multiple usages and rules with specific, "ANY", "KNOWN", and "UNKNOWN" trading partners may produce duplicate reporting records. And since Data Transformation has no real direction there will be two records for each transformation process. One for inbound and one for outbound.
To assist customers with the duplicate reporting, new fields have been added to the Management Reporting Output records to identify the trading partners from the usages and rules.
Additional Information
DB2 performance testing shows improvements with adding the TPNICKN or the MAPID selection criteria.
Ordering PTF's from the z/OS Support Web page
WDI V3.3 Server Fix Packs for AIX and Windows
Outstanding issues:
As of June 30, 2014, there are no outstanding issues.
Related Information
[{"Product":{"code":"SSFKTZ","label":"WebSphere Data Interchange"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Not Applicable","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF033","label":"Windows"},{"code":"PF035","label":"z\/OS"}],"Version":"3.3","Edition":"All Editions","Line of Business":{"code":"LOB59","label":"Sustainability Software"}}]
Was this topic helpful?
Document Information
Modified date:
19 July 2019
UID
swg27042123