Understanding the results of AsDouble(), AsFloat(), and AsInteger() transform functions

Avoid the raw character interpretation of the results of the AsDouble(), AsFloat(), and AsInteger() functions by manually converting the result to a string.

"_" : AsDouble(RowsIn.test) :"_"

to

"_" : DfloatToStringNoExp(AsDouble(RowsIn.test),2):"_"

Detailed explanation

The functions AsDouble(), AsFloat(), and AsInteger() never convert output into strings.

• Usual function scenario: result of the functions is converted to strings

  A normal transformer derivation concatenates string constants onto the beginning and end of the result of a transformer data conversion function:

  output.column=“ABC”: function(input.column):”DEF”

  In cases where the transformer conversion function returns a type that is not a character, the transformer compiler must add code to convert the result of function to a type which can be concatenated. Usually the output of function is internally converted to a string and concatenated.

  For example, the function in the derivation returns a string in the usual case. A generator stage creates a sequence of numbers. The transformer adds underscore (_) characters to the result of the absolute value of the generator input:

  "_": Abs(one_row.test):"_"

  The result is written to a sequential file. View the job diagram in InfoSphere DataStage designer:
  Figure 1. Transformer function that results in a string in InfoSphere DataStage designer.

  

  The following graphic shows the derivation in InfoSphere DataStage:

  Figure 2. Screen output showing standard transform function that results in a string.

  The following pseudo code in the .trx file is created for each transform operator:

  // Generated file to implement the _FnsAbstoStringtoSeq_Transformer transform operator. 
  // define our input/output link names 
  inputname 0 RowsIn; 
  outputname 0 seqout; 
  initialize { 
  // define our control variables 
  int8 RowRejected0; 
  int8 NullSetVar0; 
  
  // declare our intermediate variables for this section 
  ustring InterVar0_0; 
  string InterVar0_2; 
  
  // initialise constant values which require conversion 
  InterVar0_0 = "_"; 
  InterVar0_2 = "_"; 
  } 
  mainloop { 
  // declare our intermediate variables for this section 
  ustring InterVar0_1; 
  int32 InterVar0_3; 
  string InterVar0_4; //This will hold the result of the Abs() function
  
  // initialise the rejected row variable 
  RowRejected0 = 1; 
  // evaluate columns (no constraints) for link: seqout 
  InterVar0_1 = RowsIn.test; 
  seqout.S0 = ((InterVar0_0 + InterVar0_1) + InterVar0_0); 
  InterVar0_3 = RowsIn.test; 
  InterVar0_4 = abs(InterVar0_3); 
  seqout.Abs_char = ((InterVar0_2 + InterVar0_4) + InterVar0_2); 
  writerecord 0; 
  RowRejected0 = 0; 
  } 
  finish { 
  }

  In this case, the result of Abs() is an integer so an intermediate variable string int32 InterVar0_4 is generated. All the variables concatenated into sequout.Abs.char are strings.

  The sequential file output is predictable:

  "S0","Abs_char"
  "_1_","_1_"
  "_4_","_4_"
  "_7_","_7_"
  "_10_","_10_"
  "_13_","_13_"
  "_16_","_16_"
  "_19_","_19_"
  "_22_","_22_"
  "_25_","_25_"

  Both the input and the result are converted into strings for the output display.

• Scenario: using functions AsDouble(), AsFloat(), and AsInteger().

  In order to maintain compatibility with legacy versions of InfoSphere DataStage, the transformer compiler does not convert certain function results to a string. These functions are: AsDouble(), AsFloat(), and AsInteger().

  The following graphic shows a simple job which runs generated data through a transformer to a sequential file:

  Figure 3. Transform functions compared in InfoSphere DataStage designer.

  The transformer operator takes an input float and runs it through the functions AsDouble(), AsFloat(), AsInteger(), and Abs().

  The following graphic is an example of the transformer derivations:

  Figure 4. Transformer derivations using AsDouble(), AsFloat(), AsInteger(), and Abs().

  The following generated transformer code for these functions is similar to code shown above.

  For AsDouble(), AsFloat(), and AsInteger() functions:

  //
  // Generated file to implement the V0S284_FnsVSDecimaltoStringNoPeek1_KEY_Transformer transform operator. 
  // 
  // define our input/output link names 
  inputname 0 RowsIn; 
  outputname 0 seqout;
   
  initialize { 
  // define our control variables 
  int8 RowRejected0; 
  int8 NullSetVar0; 
  
  // declare our intermediate variables for this section 
  ustring InterVar0_0; 
  string InterVar0_5; 
  
  // initialise constant values which require conversion 
  InterVar0_0 = "_"; 
  InterVar0_5 = "_";
  } 
  
  mainloop { 
  
  // declare our intermediate variables for this section 
  ustring InterVar0_1; 
  int64 InterVar0_2; // Used for AsInteger() 
  sfloat InterVar0_3;// Used for AsFloat() 
  dfloat InterVar0_4;// Used for AsDouble() 
  int32 InterVar0_6; 
  string InterVar0_7;// Used for Abs() 
  
  // initialise the rejected row variable 
  RowRejected0 = 1; 
  
  // evaluate columns (no constraints) for link: seqout 
  InterVar0_1 = RowsIn.test; 
  seqout.S0 = ((InterVar0_0 + InterVar0_1) + InterVar0_0); 
  InterVar0_2 = RowsIn.test; 
  seqout.AsInteger_char = ((InterVar0_0 + InterVar0_2) +InterVar0_0); 
  InterVar0_3 = RowsIn.test; 
  seqout.AsFloat_char = ((InterVar0_0 + InterVar0_3) +InterVar0_0); 
  InterVar0_4 = RowsIn.test; 
  seqout.AsDouble_char = ((InterVar0_0 + InterVar0_4) +InterVar0_0); 
  InterVar0_6 = RowsIn.test; 
  InterVar0_7 = abs(InterVar0_6); 
  seqout.Abs_char = ((InterVar0_5 + InterVar0_7) + InterVar0_5); 
  writerecord 0; 
  RowRejected0 = 0; 
  } 
  
  finish { 
  }

  Note: Below is an excerpt of a sequential file which shows that outputs from AsDouble(), AsFloat(), and AsInteger() functions are single ASCII characters whereas output from an Abs() function (a usual-case function) is interpreted as a string. Note how sending the raw bits for character number 10 (which is 0xA, a Line Feed character) puts new lines in the displayed output, and how the raw code produces the glyph for a non-printing character for characters 1,4, and 7.

  Figure 5. Sequential file comparing function output.