PH29619: Truncated data and NO_VALUE column processing in Column Map PROCEDUREs with LEGACY TABLES

Obtain the fix for this APAR.

APAR status

• Closed as program error.

Error description

• This APAR addresses various defects related to processing of
  EXTRACTED IMS or other non-DB2 data.
  
  Customer is processing EXTRACTED IMS or other non-DB2 data.
  The Legacy Table (LT), which describes the record layout, does
  not properly reflect the record's attributes by using a fixed
  length layout (while it has variable length) and contents
  (the records may be shorter than defined via LT).
  
  Because of that, Optim marks some of the columns as missing and
  having the NO_VALUE attribute when the record read for
  processing is shorter than expected.
  
  The following problems will be addressed by this APAR:
  1.The EXTRACT Browse function may show garbage for some columns
  when shorter records follow records with more data on EXTRACT
  input.
  2.When the variable length records of VSAM or sequential origin
  are being inserted back in to a variable length destination file
  then the records will be expanded to the full length of the LT
  used to describe the destination table/file.
  3.When the EXTRACT is being processed by the CONVERT function
  then the converted data may grow significantly in size (compared
  to the EXTRACT) because the records will be expanded to the full
  length of the LT used to describe the destination table/file.
  4.When customer uses a Column Map PROCEDURE (CM PROC) to process
  the records, and uses getcolumnasbytes/getcolumnvalue functions,
  the returned function value is a nil (NULL) indicating that
  no value was available for the requested column in the record
  being processed.
  Optim does not provide a function to set the destination column
  to the same state of having "no value" since LTs do not allow
  for columns to accept a NULL value (in the meaning of DB2).
  Furthermore Optim requires that a destination column, on which
  the CM PROC has been defined, always receives a value.
  This may force the user to assign a value which does not reflect
  the NO_VALUE state of the source value.
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED: All Optim users working with non-DB2 data    *
  *                 (VSAM, sequential and IMS).                  *
  ****************************************************************
  * PROBLEM DESCRIPTION: Optim may incorrectly process Legacy    *
  *                      Table data with records shorter than    *
  *                      the record definition within the LT.    *
  *                      Optim does not provide for a function   *
  *                      which may set a NO_VALUE state on a     *
  *                      column within an LT.                    *
  ****************************************************************
  See Error description.
  The problem is a current product design limitation.
  

Problem conclusion

• Optim processing logic in respect to LTs incorrectly describing
  the record structure and length of data will be corrected and
  enhanced.
  Optim will notify the user during EXTRACT file creation that it
  detected records with data shorter than the LT definition. In
  addition, the following new messages will be issued when the
  trailing column being truncated is not of the type of CHARACTER,
  GRAPHIC or BINARY.
  
  WARNING: Table (1) had 2 records with trailing column value
           truncated mid-value.
           The trailing column was of data type other than
           CHAR/GRAPHIC/BINARY.
           The truncated value has been discarded.
           Consider updating the Legacy Table definition and using
           an ODO/MAX_OCCURS array.
  
  Optim processing logic will be enhanced and allow for a column/
  field value in a LT to be set to a NULL value to indicate the
  lack of a value (NO_VALUE) condition.
  A column from a LT may have the state of NO_VALUE when the
  associated LT record does not contain a value for it. This may
  occur for ODO type values when not all values of an array have
  been provided, or for variable length records when the record is
  shorter than its maximum documented length and some fields may
  not receive a value when such record is being extracted.
  LT fields/columns with the NO_VALUE state can be currently
  processed only via CM EXITs (type 2) and CM PROCs and
  getcolumnvalue() type functions.
  Such functions would return a nil for a column with the NO_VALUE
  attribute. However the user had no means to set a destination LT
  column to NO_VALUE and had been required to set it to some other
  value, possibly losing the the NO_VALUE attribute.
  This enhancement will allow for the setcolumnvalue() family of
  functions to set a LT column a NO_VALUE state by assigning it a
  nil (NULL) value via a CM_PROC.
  
  Example of use:
  
   PROC LOCAL                            RET_ASTT_OPTN_PAYEE_STATE
   1 state = optim.source.getcolumnvalue()
   2 if state == nil then
   3   optim.print("Column value is: NO_VALUE")
   4   -- We have a field in the LT which had NO_VALUE because
   5   -- it does not exist in the record which is too short
   6   -- We still allow an assignment to it because it could
   7   -- be included in an ODO and the occurs value could have
   8   -- been expanded. And we have enough room in the record for
          it.
   9   -- Note that we actually require that the target column be
          set
  10   -- and we report an error for columns having no value
          assigned.
  11   -- After APAR PH29619/PH29667 the following value
          assignment
  12   -- will be accepted which assigns a NO_VALUE state to the
  13   -- destination column. This is equivalent to "regular"
          value
  14   -- assignment but at the moment we won't do anything with
          this
  15   -- flag, just suppress the "value not assigned error".
  16   optim.target.setcolumnvalue(nil)
  17 else
  18   -- We expect the nil above because the field has NO_VALUE
  19   -- in the record which is shorter than expected.
  20   -- We should never see this with the short record.
  21   optim.print("Column value is: >" .. state .. "<")
  22   optim.target.setcolumnvalue(state)
  23 end
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  PH29667

Modules/Macros

• FOP00003 FOP00103 FOP00203 FOP00303 FOP00B   FOP01B   FOP05B
  FOP07B   FOP08B   FOP09B   FOP0AB   FOP0BB   FOP0ITKA FOP1ODRV
  FOP1OSFN FOP3DTPR FOP4IPMN FOP4PEMN FOP4PMMN FOP4SVC  FOP4TRCE
  FOP4XCCN FOP4XEML FOP4XSSN FOP9APIM FOP9GINF FOPCFPMN FOPCXLIB
  FOPHSTM  FOPIO    FOPMADDE FOPMAFCN FOPMAIN  FOPMAMAI FOPMARCN
  FOPMCDRT FOPMCENU FOPMCMAI FOPMCOFN FOPMCOLA FOPMCOLP FOPMDBMN
  FOPMDDDL FOPMDMAI FOPMDPRC FOPMDXTR FOPMECMN FOPMEMAI FOPMIXMM
  FOPMMCMN FOPMMCMR FOPMMJBS FOPMMUNU FOPMMUTL FOPMSELM FOPMTMDB
  FOPMUXIM FOPODPP  FOPOMN   FOPRS9   FOPTDS25 FOPUMCBT FOPUUNLD
  FOPVOU   FOPZFBMN FOPZFMMN FOPZMAIN FOPZSUB
  

Fix information

• Fixed component name

  OPTIM MOVE FOR

• Fixed component ID

  5655V0700

Applicable component levels

• RB70 PSY UI72393

     UP20/11/10 P F011

Fix is available

• Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.