Mapping Data in Flow Services

Because systems rarely produce data in the exact format that other systems need, you commonly need to build flow services that perform data transformations. Data transformation resolves differences in the way data is represented within documents that applications and systems exchange. In Designer, data transformations can be accomplished by mapping data. By mapping, you can accomplish the following types of transformations:

  • Name transformations. This type of transformation resolves differences in the way data is named. For example, one service or document format might use telephone as the name of the variable for telephone number information and another might use phoneNumber. When you perform name transformations, the value and position of a variable in the document structure remains the same, but the name of the variable changes.
  • Structural transformations.This type of transformation resolves differences in the data type or structure used to represent a data item. For example, one service or document format might put the telephone number in a String called telephone, and the next may expect to find it nested in a Document named customerInfo. When you perform structural transformations, the value of the variable remains the same, but the data type or position of the variable in the Document structure changes.
  • Value transformations.This type of transformation resolves differences in the way values are expressed (for example, when systems use different notations for values such as standard codes, units of currency, dates, or weights and measures). When you perform value transformations, the name and position of the variable remain the same, but the data contained in the variable changes. For example, you can change the format of a date, concatenate two Strings, or add the values of two variables together.

When you build flow services or convert between document formats, you may need to perform one, two, or all of the above types of data transformation. The webMethods flow language provides two ways for you to accomplish data transformations between services and document formats in the pipeline: you can map variables to each other (create links) or you can insert transformers, which are services invoked within a MAP step.

What Does the Pipeline View Contain?

The Pipeline view offers a graphical representation of all of your data through which you can map data and inspect the contents of the pipeline. You use the tools on this view to route variables (data) between services or between document formats.

The Pipeline view displays the pipeline for invoked services (INVOKE steps) or MAP steps in a flow service. The contents of the Pipeline view are different for INVOKE steps than for MAP steps.

Pipeline View for an INVOKE Step

For an INVOKE step, the Pipeline view depicts two stages of the pipeline with respect to the selected service in the editor.

Figure 1. Pipeline view for an INVOKE step
This stage... Represents...
1 The expected state of the pipeline just before the selected service executes.

Pipeline In depicts the set of variables that are expected to be in the pipeline before the service executes (based on the declared input and output parameters of the preceding services).

Service In depicts the set of variables the selected service expects as input (as defined by its input parameters).

In the Pipeline view, you can insert “pipeline modifiers” at this stage to adjust the contents of the pipeline to suit the requirements of the service. For example, you can link variables, assign values to variables, drop variables from the pipeline, or add variables to the pipeline. Modifications that you specify during this stage are performed immediately before the service executes at run time.

2 The expected state of the pipeline just after the service executes.

Service Out depicts the set of variables that the selected service produces as output (as defined by its output parameters).

Pipeline Out depicts the set of variables that are expected to be in the pipeline after the service executes. It represents the set of variables that will be available to the next service in the flow. If the selected service (INVOKE step) is the last step in the flow service, Pipeline Out displays the output variables for the flow service (as declared on the Input/Output tab).

In the Pipeline view, you can insert “pipeline modifiers” at this stage to adjust the contents of the pipeline. For example, you can link variables, assign values to variables, drop variables from the pipeline, or add variables to the pipeline. Modifications that you specify during this stage are performed immediately after the service executes at run time.

Note: Designer displays small symbols next to a variable icon to indicate validation constraints. Designer uses to indicate an optional variable. Designer uses the ‡ symbol to denote a variable with a content constraint. Designer also uses to indicate that the variable has a default value that can be overridden assigned to it and to indicate that the variable has a null value that cannot be overridden assigned to it. A combination of the and symbols next to a variable icon indicates that the variable has a fixed default value that is not null and cannot be overridden. For information about applying constraints to variables, see About Variable Constraints.

Pipeline View for a MAP Step

For a MAP step, the Pipeline view displays a single stage of the pipeline. The Pipeline view contains two sets of variables: Pipeline In and Pipeline Out. Between these sets of variables, the Pipeline view contains a column named Transformers.

Figure 2. Pipeline view for a MAP step
  • The Pipeline In column represents input to the MAP step. It contains the names of all of the variables in the pipeline at this point in the flow.
  • The Transformers column displays any services inserted in the MAP step to complete value transformations. For more information about invoking services in a MAP step, see Working with Transformers.
  • The Pipeline Out column represents the output of the MAP step. It contains the names of variables that will be available in the pipeline when the MAP step completes.

When you first insert a MAP step into your flow, Pipeline In and Pipeline Out are identical. However, if the MAP step is the only step in the flow service or is the last step in the flow service, Pipeline Out also displays the variables declared as output in the flow service.

Scrolling in Pipeline View

About this task

By default, you scroll vertically or horizontally through the entire Pipeline view. However, you can enable horizontal and vertical scrolling for each column in Pipeline view. Independent scrolling is especially useful when mapping a large amount of data in the Pipeline view.

To enable independent scrolling in Pipeline view

Procedure

Right-click anywhere inside the Pipeline view and select Enable independent scrolling to scroll through each column in Pipeline view horizontally and vertically independent of other columns.
Tip: While scrolling through a large amount of data, if you do not want Designer to display the links if the source or target variables are not visible, right-click anywhere inside the Pipeline view and select Hide links if variables are not visible.

Viewing Full Namespace Path of Referenced Document Types

About this task

By default, Designer displays only the names of the document reference or document reference list variables in Pipeline view. However, you can choose to view or hide the full namespace path of the referenced document types in Pipeline view. If you choose to view the full namespace path of the referenced document types in Pipeline view, they appear in parenthesis after the name of the document reference or document reference list variables, for example, contactInfo (DocTypesAndSpecs:address)

The Show Referenced Document Type Name setting applies to:
  • All document reference or document reference list variables in Pipeline view and not just the selected nodes.
  • All MAP and INVOKE steps of all flow services.

This setting remains in effect even after you shutdown and restart Integration Server.

Note: You can also use the Flow Service Editor preferences page to view or hide the full namespace path of the referenced document types in Pipeline view.

To view or hide the full namespace path of the referenced document types in Pipeline view

Procedure

Right-click anywhere inside the Pipeline view and select Show Referenced Document Type Name.

Printing the Pipeline

About this task

The following procedure describes how to use the View as HTML command to produce a printable version of the pipeline.

To print the pipeline

Procedure

  1. In the Package Navigator view, open the flow service for which you want to print the pipeline.
  2. In the editor, select the INVOKE or MAP step for which you want to print the pipeline.
  3. Scroll or resize the Pipeline view to display the portion of the pipeline you want to view as HTML.
  4. Right-click anywhere inside the Pipeline view and click View as HTML.

    Designer creates an HTML page and displays it in your default browser.

  5. Use your browser's print command to print the pipeline.

Basic Mapping Tasks

Basic mapping tasks are the tasks you perform to manage the pipeline contents and the values of variables in the pipeline. In the Pipeline view, you can perform the following basic mapping tasks:

  • Link variables to each other. You can copy the value of a variable in one service or document format to a variable in another service or document format.
  • Assign values to variables. You can hard code variable values or assign a default value to variables.
  • Drop variables from the pipeline. You can remove pipeline variables that are not used by subsequent services in a flow.
  • Add variables to the pipeline. You can add variables that were not declared as input or output parameters of the flow service. You can also add input and output variables for services that the flow service invokes (internally invoked services).

About Linking Variables

When you want to copy the value of a variable in a service or document format to another variable, you link the variables. Designer connects service and pipeline variables in the Pipeline view with a line called a link. Creating a link between variables copies the value from one variable to another at run time.

Within a flow, Designer implicitly links variables whose names are the same and whose data types are compatible. For example, the service in the following flow takes a variable called AcctNumber. Because a variable by this name already exists in Pipeline In, it is automatically linked to the AcctNumber variable in Service In. Designer connects implicitly linked variables with a gray link.

Figure 3. Implicit links between pipeline and service variables
Note: The Pipeline view does not display implicit links for a MAP step.

In cases where the services in a flow do not use the same names for a piece of information, use the Pipeline view to explicitly link the variables to each other. Explicit linking is how you accomplish name and structure transformations required in a flow. Designer connects explicitly linked variables with a solid black line.

On the input side of the Pipeline view, use to link a variable from the pipeline to the service. In the following example, the service expects a value called OrderTotal, which is equivalent to the pipeline variable BuyersTotal (that is, they are simply different names for the same data). To use the value of BuyersTotal as the value for OrderTotal, you “link” the pipeline variable to the service using .

At run time, the server will copy the value from the source variable (BuyersTotal) to the target variable (OrderTotal) before executing the service.

Figure 4. Linking the pipeline to service input
Important: Do not link variables with different Object constraints. If you link variables with different Object constraints and input/output validation is selected, the run-time result is undefined.

All the output variables that a service produces are automatically placed in the pipeline. Just as you can link variables from the Pipeline In stage to a service’s input variables, you can link the output from a service to a different variable in Pipeline Out.

In the following example, a variable called TransNumber is linked to the field Num in a Document called TransactionRecord. At run time, the server will copy the value of TransNumber to Num, and both TransNumber and Num will be available to subsequent services in the flow.

Figure 5. Linking service output to the pipeline

Creating a Link Between Variables

About this task

When you link variables in the pipeline, keep the following points in mind:
  • The variable that you are linking from is the source. For example, when you link a variable in Pipeline In to one in Service In, the Pipeline In variable is the source. When you link a variable in Service Out to one in Pipeline Out, the Service Out variable is the source.
  • The variable you are linking to is the target. For example, when you link a variable in Pipeline In to one in Service In, the Service In variable is the target. When you link a variable in Service Out to one in Pipeline Out, the Pipeline Out variable is the target.
  • A Service In variable can be the target of more than one link only if you use array indexing or if you place conditions on the links to the variable.
  • By linking variables to each other, you are copying data from the source variable to the target variable. (Documents, however, are copied by reference. For more information, see What Happens When Integration Server Executes a Link?.)
  • Target variables can be connected to only one source variable. After you draw a link to a target variable, you cannot draw another link to the target variable. (Two exceptions to this rule involve array variables and conditional links. For more information about linking array variables, see Linking to and from Array Variables in the Pipeline. For more information about placing conditions on links between variables, see Linking Variables Conditionally.
  • You cannot create a link to a variable if you already assigned a value to a variable.
  • After a link executes, both the source and target variables exist in the pipeline. The target variable does not replace the source variable.
  • You cannot create a link to a variable if the variable has a fixed null or default value assigned to it. Designer uses the symbol next to the variable icon to indicate that the variable has a fixed value that you cannot override by linking it to another variable.

To create a link between variables

Procedure

  1. In the flow service editor, select the INVOKE or MAP step containing the variables you want to link.
  2. Open the Pipeline view.
  3. If you want to create a link between a variable in Pipeline In and one in Service In, do the following:
    1. In Pipeline In, click the pipeline variable you want to use as the source variable.
    2. In Service In, click the input variable you want to use as the target variable.
    3. Click on the Pipeline view toolbar.
  4. If you want to create a link between a variable in Service Out and one in Pipeline Out, do the following:
    1. In Service Out, click the output variable you want to use as the source variable.
    2. In Pipeline Out, click the pipeline variable you want to use as the target variable.
    3. Click on the toolbar.
  5. Click File > Save.

Results

Notes:
  • If the variable types are incompatible and cannot be linked to one another, Designer prevents you from creating a link between the variables and displays a message stating that the operation is not allowed.
  • If you created a link to or from an array variable, you must specify which element in the array you are linking to or from. For more information about array linking, see Linking to and from Array Variables in the Pipeline.
  • If you want to place a condition on the execution of the link, see Linking Variables Conditionally.
  • Do not link variables with different Object constraints. If you link variables with different Object constraints and input/output validation is selected, the run-time result is undefined.
Tip: You can also use your mouse to link variables to one another. To do this, select the source variable and drag your mouse to the appropriate target variable.

What Happens When Integration Server Executes a Link?

When executing a link between variables at run time, Integration Server does one of the following:

  • Copies the value from the source variable to the target variable. For example, when you link a source String variable to a target String variable, Integration Server copies the value of the source String to the target String. This is called “copying by value.”
  • Creates a reference to the source variable and uses the reference as the value of the target variable. For example, when executing a link between a source Document and a target Document, Integration Server creates a reference to the source Document value and uses the reference as the value of the target Document. This is called “copying by reference.”

Integration Server copies by value when the source or target variable is a String. (An exception to this behavior is that when executing a link from a String to an Object, the Integration Server copies by reference.)

When executing links between all other types of variables, Integration Server copies by reference. Copying by reference significantly reduces the memory and time required for executing a link at run time.

When a value is copied by reference, any changes you make to the value of the source variable in subsequent flow steps affect the target variable. This is because the value of the source variable is the value of the target variable. The target variable does not contain a copy of the source variable value. If, in a later flow step, you used to assign a value to the source variable, you would be changing the value of the target variable as well. (The target variable references the value of the source variable.)

Example of Copying By Reference

The following images show a series of MAP steps in a flow service. In this example, the value of the source variable is changed after the link to the target variable executes. This action changes the value of the target variable as well.

Figure 6. Step 1: The value of String1 is set to “original value”
Figure 7. Step 2: Document1 is linked to Document2
Figure 8. Step 3: The value of String1 is changed to “modified” after the link executes

When this flow service executes, it returns the following results.

Figure 9. Results of flow service

In Step 3, the value of the String1 in Document1 was set to “modified.” However, the value of String1 in Document2 changed also. This is because in Step 2 of the flow service, the value of Document1 was copied to Document2 by reference. Changes to the value of Document1 in later flow steps also change the value of Document2.

Preventing Pipeline Values from Being Overwritten

To prevent the value of the target variable from being overwritten by changes to the value of the source value in subsequent steps in the flow service as demonstrated in Example of Copying By Reference, you can do one of the following:

  • When working with Document variables, link each child of the Document variable individually. This method can be time consuming and might significantly increase the memory and time required to run the service. However, this might be the best approach if the target Document variable needs only a few values from the source Document variable.
  • After you link the source variable to a target variable, use the Drop modifier to drop the source variable. Only the target variable will have the reference to the data. This method ensures that the value of the target variable will not be overwritten in a subsequent step, but does not increase the memory and time required to execute the service.
  • Create a service that performs a copy by value. Insert this service (as an INVOKE step or as a transformer) and link the variables to the service instead of linking them to each other. (In the case of Document variables, you could create a Java service that clones the IData object underlying the Document.) In situations where you link one Document variable to another, using a “cloning” service would require less time than linking the contents of a Document variable field by field.

Linking to Document and Document List Variables

When working with Document variables in the pipeline, you can link a source variable to the Document variable or to the children of the Document variable. Keep the following points in mind when linking to or from Document, Document List, Document Reference, or Document Reference List variables:

  • A Document (or a Document List) and its children cannot both be targets. After a Document or Document List is the target of a link, its children cannot be the targets of links.
  • After the child variable of a Document or Document List is the target of a link, the parent Document or Document List cannot be a target of a link.
  • If you link from a Document variable to another Document variable, the structure of the source Document variable overwrites the structure of the target Document variable.
  • You cannot link a nested Document List to a target Document List when the Document Lists have different sizes. A nested Document List is one that is contained within a parent Document List. Document Lists are considered to have different sizes when they have a different number of entries within the lists. If you need to move values from the source Document List to the target, create user code that uses a LOOP flow step to assign values from the source to the target one by one.
  • When a Document Reference or Document Reference List refers to an IS document type that contains identically named variables that are of the same data type and both identically named variables are assigned a value or are linked to another variable, Integration Server might not maintain the order of the document contents in the pipeline when the service executes. For example, Integration Server might group all of the identical variables at the end of the document. To prevent the change in the order of document contents, set default values for the identically named variables. To do this, insert a MAP step in the service before the step in which you want link or assign a value to the variables. In the MAP step, under Pipeline Out, select the Document Reference variable and click on the Pipeline view toolbar. In the Enter Input for dialog box, assign default values to the identically named variables.

Linking Variables of Different Data Types

In the Pipeline view, you can link different, but compatible, data types to one another. For example, you could link a String value called AccountNumber to a String List called Accounts. At run time, the server automatically performs the structural transformation necessary to link the data in AccountNumber to Accounts. (In this case, the transformation will result in a single-element String List.) By linking different data types to one another, you can perform structural transformations.

If you link variables of different data types, keep the following points in mind:

  • Not all data types can be linked to one another. You cannot link a Document (IData object) to a String, for instance. If two data types are incompatible, Designer will not allow you to link them to each other.
  • You can only link a variable to another variable of the same primitive type. The primitive type refers to the data type of the variable when all dimensionality is removed. For example, the primitive type for a String List or a String Table would be String. Two exceptions to this rule are the following:
    • Any variable can be linked to an Object or an Object List variable
    • An Object can be linked to any data type.

    If there is a type mismatch between the Object or Object List and the other variable at run time, Integration Server does not execute the link.

  • Object and Object List variables constrained with an assigned Java class should be linked only to other Object and Object List variables of the same Java class or to Object and Object List variables of unknown type. Although Designer permits a link between constrained Objects with different Java classes, the run-time result is undefined. For more information about specifying Java classes for Objects, see Java Classes for Objects.
  • When you link between scalar and array variables, you can specify which element of the array variable you want to link to or from. Scalar variables are those that hold a single value, such as String, Document, and Object. Array variables are those that hold multiple values, such as String List, String Table, Document List, and Object List. For example, you can link a String to the second element of a String List. Alternatively, you can link the second element in a String List to a String.
  • When you link between scalar and array variables and you do not specify which element in the array variable that you want to link to or from, Designer uses the default behavior to determine the value of the target variable.

Converting a String List to a Document List in the Pipeline

You can convert a String List to a Document List in the pipeline by mapping a String List to a String in a Document List. In the following image, aList is the String List you want to convert to a Document List. The variable documentList is the Document List to which you want to copy the values contained in the String List. documentList has a String child aString. To convert the String List to a Document List, link aList to aString.

Figure 10. Converting a String List to a Document List

Converting Two String Lists to a Document List in the Pipeline

Two String Lists can be combined into one Document List in the pipeline by linking each String List to a String nested in a Document List. For example, suppose that you had String List variables named aList and bList, and documentList had two String children named aString and bString. You could combine the two String Lists by linking aList to aString and bList to bString.

Figure 11. Converting two String Lists to a Document List
Tip: You can also convert a String List to a Document List (IData[ ] object) by invoking the built-in service pub.list:stringListToDocumentList. You can insert the service as an INVOKE step or as a transformer. For more information about transformers, see Working with Transformers. For more information about built-in services, see the webMethods Integration Server Built-In Services Reference.

Linking to and from Array Variables in the Pipeline

When you link to or from an array variable (String List, String Table, Document List, or Object List), you can specify which element in the array you want to link to or from. After you link the variables, you specify the index that represents the position of the element in the array.

  • For String Lists and Object Lists, you can specify the index for the list element you want to link. For example, you can link the third element in a String List to a String.
  • For String Tables, you can specify the row and column indexes for the cells you want to link. For example, you can link the value of the element in the third column of the second row of a String Table to a String.
  • For Document Lists, you can specify the index for the Document that you want to link. For example, you can link the second Document in a Document List to a Document variable.
  • For a variable in a Document List, you can specify the index of the Document that contains the value that you want to link to or from. For example, if the Document List POItems contains the String ItemNumber, you can link the ItemNumber value from the second POItems Document to a String variable.

For example, suppose that a buyer’s address information is initially stored in a String List. However, the information might be easier to work with if it were stored in a Document. To map the information in the String List to a Document, create a link between the String List and each field in the Document. Then, specify an index value for each link. In the following pipeline, the elements in buyerAddress String List are mapped to the address Document.

Figure 12. You can specify an index value when linking to or from an array variable
Note: Designer uses blue links in the Pipeline view to indicate that properties (conditions or index values for arrays) have been applied to the link between variables.

Creating a Link to or from an Array Variable

About this task
When you are linking to or from an array variable, keep the following points in mind:
  • To link to or from an element in an array variable, you need to know the index for the element’s position in the array. Array index numbering begins at 0 (the first element in the array has an index of 0, the second element has an index of 1, and so on).
  • To dynamically specify the index, you can set the index to the value of a pipeline variable. The variable you specify must be a String. To use a pipeline variable, specify the variable name enclosed in percent signs (%). For example, if the pipeline contains the variable itemNumber that will contain the index you want to use at run time, specify %itemNumber% for the index. For the link to execute successfully at run time, the value of the variable must be a non-negative integer.
  • If you link to an array variable and specify an index that does not exist, Designer increases the length of the array to include the specified array index. For example, suppose that a String List has length 3. You can link to the String List and specify an index of 4; that is, you can link to the fifth position in the String List. At run time, the Integration Server increases the length of the String List from 3 to 5.
  • Each element in an array can be the source or target of a link; that is, each element in the array can be the start or end of a link. For example, if a source String List variable contains three elements, you can link each of the three elements to a target variable.
  • If the source and target variables are arrays, you can specify an index for each variable. For example, you can link the third element in a source String List to the fifth element in target String List.
  • If you do not specify an array index for an element when linking to or from arrays, the default behavior of the Pipeline view will be used. For information about the default behavior of the Pipeline view, see Default Pipeline Rules for Linking to and from Array Variables.
  • If you are linking to or from a String Table, you need to specify an index value for the row and column.
  • When you link a Document or Document List variable to another Document or Document List variable, the structure of the source variable determines the structure of the target variable. For more information, see Linking to Document and Document List Variables.
  • At run time, the link (copy) fails if the source array index contains a null value or if you specify an invalid source or target index (such as a letter or non-numeric character). Integration Server generates journal log messages (at debug level 6 or higher) when links to or from array variables fail.

The following procedure explains how to link to or from an array variable.

To create a link to or from an array variable

Procedure
  1. Create a link between the variables using the procedure described in Creating a Link Between Variables.
  2. In Pipeline view, click the link that connects the variables.
  3. In the Properties view, click the Indices value and click . The Link Indices dialog box appears.
  4. If the source variable is an array variable, under Source, type the index that contains the value you want to link. If the source variable is a String Table, you need to specify a row index and a column index.
  5. If the target variable is an array variable, under Destination, type the index to which you want to link the source value. If the target variable is a String Table, you need to specify a row index and a column index.
  6. Click OK.
Results
Tip: You can also open the Link Indices dialog box by selecting the link between the variables and clicking on the Pipeline view toolbar.

Default Pipeline Rules for Linking to and from Array Variables

When you create links between scalar and array variables, you can specify which element of the array variable you want to link to or from. Scalar variables are those that hold a single value, such as String, Document, and Object. Array variables are those that hold multiple values, such as String List, String Table, Document List, and Object List. For example, you can link a String to the second element of a String List.

If you do not specify which element in the array variable that you want to link to or from, Designer uses the default rules in the Pipeline view to determine the value of the target variable. The following table identifies the default pipeline rules for linking to and from array variables.

If you link… To… Then…
A scalar variable An array variable that is empty (the variable does not have a defined length) The link defines the length of the array variable; that is, it contains one element and has length of one. The first (and only) element in the array is assigned the value of the scalar variable.
If you link… To… Then…
A scalar variable An array variable with a defined length The length of the array is preserved and each element of the array is assigned the value of the scalar variable.
If you link… To… Then…
An array variable A scalar variable The scalar variable is assigned the first element in the array.
If you link… To… Then…
An array variable An array variable that does not have a defined length The link defines the length of the target array variable; that is, it will be the same length as the source array variable. The elements in the target array variable are assigned the values of the corresponding elements in the source array variable.
If you link… To… Then…
An array variable An array variable that has a defined length The length of the source array variable must equal the length of the target array variable. If the lengths do not match, the link will not occur. If the lengths are equal, the elements in the target array variable are assigned the values of the corresponding elements in the source array variable.
No link occurs.

A source variable that is the child of a Document List is treated like an array because there is one value of the source variable for each Document in the Document List. For example:

If you link… To…
Where the value of DocumentList1 is... Then the value of StringList1 is…

Deleting a Link Between Variables

About this task

When you delete link in Pipeline view, the variables are no longer linked. Designer also deletes any properties you applied to the link.

To delete a link between variables

Procedure

  1. In the flow service editor, select the INVOKE or MAP step containing the variables with the link you want to delete.
  2. In the Pipeline view, select the link that you want to delete.
  3. Click Edit > Delete.

What to do next

Tip: You can also delete a link by selecting it and then pressing the DELETE key.

Linking Variables Conditionally

You can place conditions on the links you draw between variables. At run time, Integration Server evaluates the condition and executes the link (copies the value) only if the condition evaluates to true.

A condition consists of one or more expressions that you write using the syntax that Designer provides. An expression can check for the existence of a variable in the pipeline, check for the value of a variable, or compare a variable to another variable. For example, in the following service, you might want to link the BuyersTotal variable in Pipeline In to the OrderTotal variable in Service In only if the BuyersTotal has a value that is not null. After you link two variables, you would edit the properties and add the condition that needs to be true.

Figure 13. A blue link indicates that a condition is applied to the link connecting the variables

Designer uses a blue link in the Pipeline view to indicate that properties (that is, conditions or index values for arrays) have been applied to a link between variables.

Note: You cannot add conditions to the links between implicitly linked variables.

Linking Multiple Source Variables to a Target Variable

By applying conditions to the links between variables, you can link more than one source variable to the same target variable. When you draw more than one link to the same target variable, at most, only one of the conditions you apply to the links can be true at run time. The conditions must be mutually exclusive.

At run time, Integration Server executes all conditional links whose conditions evaluate to true. If more than one conditional link to the same target variable evaluates to true, the value of the target variable will be the result of whichever link executes last. Because the order in which links are executed at run time is not guaranteed, the final value of the target variable may vary.

Tip: If the conditions for links to the same target variable are not mutually exclusive, consider using a flow service containing a BRANCH step instead. In BRANCH steps, child steps are evaluated in a top to bottom sequence. Integration Server executes the first child step that evaluates to true and skips the remaining child steps. For more information about the BRANCH step, see The BRANCH Step.

Applying a Condition to a Link

About this task
Keep the following points in mind when making a conditional link:
  • You can only add conditions to the links between explicitly linked variables. You cannot add conditions to the links between implicitly linked variables.
  • When drawing more than one link to the same target variable, make sure that the conditions assigned to each link are mutually exclusive.
  • You can temporarily disable the condition placed on a link. For more information, see Disabling and Enabling Conditions.

To apply a condition to the link between variables

Procedure
  1. Create a link between the variables using the procedure described in Creating a Link Between Variables.
  2. In Pipeline view, click the link that connects the variables.
  3. In the Properties view, set the Evaluate copy condition property to True.
  4. In the Copy condition property text box, type the condition you want to place on the link. For information about the syntax used in conditions, see Conditional Expressions.

About Assigning Values to Pipeline Variables

You can assign values to variables in Service In or Pipeline Out using on the Pipeline view toolbar. When you assign a value to a variable, you can:

  • Explicitly “hard code” a specific value in a variable.
  • Initialize a set of input variables by assigning values to all of the input variables.
  • Assign a default value to a variable. That is, a value that is only assigned if the variable is null at run time.
  • Assign a variable the value of another pipeline variable by referencing the variable. You might do this if you wanted to derive the default variable value from another variable in the pipeline at run time.
  • Assign a variable the value of a global variable. You might do this for values that will change after you deploy a solution such as connection credentials. Instead of changing the value assigned to the pipeline variable in each service that establishes a connection, you change the value once in the global variable definition. For more information about global variables, see Assigning Global Variables to Pipeline Variables

By using to assign a value to a variable, you instruct Integration Server to write a specific value to that variable at run time. This action occurs just before the selected service is executed (if you assign the value to a variable in Service In) or immediately after the selected service is executed (if you assign the value to a variable in Pipeline Out).

Assigning a Value to a Pipeline Variable

About this task

You can assign values to variables that are in Service In or Pipeline Out when the variable is not linked or when the variable is only implicitly linked to another value in the pipeline.

You cannot assign values to:

  • Variables that are explicitly linked to another value in the pipeline
  • Variables that have been dropped from the pipeline
  • Object variables constrained as a byte [ ]
  • Unconstrained Objects (Objects of unknown type)
  • An XML document variable or an XML document list variable
  • A variable that has a fixed null or default value assigned to it. Designer uses the symbol next to the variable icon to indicate that the variable has a fixed value that you cannot override by linking it to another variable.

To assign a value to a pipeline variable

Procedure

  1. In the flow service editor, select the INVOKE or MAP step containing the variable you want to alter.
  2. In Pipeline view, select the variable to which you want to assign a value. The variable must be in either Service In or Pipeline Out.
  3. Click on the Pipeline view toolbar.

    Designer displays the Enter Input for dialog box.

  4. Assign values using the Enter Input for dialog box. For specific information for how to assign a value based on a variable’s data type, see one of the following:

Results

Notes:
  • The Include empty values for String Type checkbox is disabled when assigning values to pipeline variables of type String, String List, String Table, Document, Object, and Object List. It is available only when assigning values to Document List variables. For more information, see Specifying Values for a Document List Variable.
  • The checkboxes next to each element in the tree are disabled when assigning values to pipeline variables of type String, String List, String Table, Document, Object, and Object List. The checkbox is only enabled for top-level Document variables within a Document List and is used along with the Include empty values for String Type checkbox. For more information, see Specifying Values for a Document List Variable.
  • The Perform pipeline variable substitution checkbox indicates whether you want Integration Server to perform pipeline variable substitution at run time. To use a variable when assigning a String value, you type the name of the pipeline variable enclosed in % symbols (for example, %Phone%). If you specify a pipeline variable enclosed in % symbols for a String value, you must select the Perform pipeline variable substitution checkbox for the variable substitution to occur.
  • The Perform global variable substitution checkbox indicates whether you want Integration Server to perform global variable substitution at runtime.

    To use a global variable when assigning a String value, you type the name of the global variable enclosed in % symbols (for example, %myFTPUsername%). When Integration Server encounters global variable substitution during service execution, Integration Server first looks for the global variable in the same package as the service. If not found, Integration Server looks for the global variable at the server level. You can override this default behavior by specifying whether the service must use a server-level or a package-level global variable. To use a server-level global variable, prefix the variable name like this: %server:variablename%. To use a package-level global variable, prefix the variable name like this: %package:variablename%. For more information, see Assigning Global Variables to Pipeline Variables.

    If you specify a global variable enclosed in % symbols for a String value, you must select the Perform global variable substitution checkbox for the variable substitution to occur.
  • If a pipeline variable and global variable have the same name and you select both Perform global variable substitution and Perform pipeline variable substitution, Integration Server uses the value of the pipeline variable.
  • The Overwrite pipeline value checkbox indicates whether you want Integration Server to use the value you specify even when the variable has a pipeline value at run time.
    • Select the checkbox to have Integration Server always use the value you specify.
    • Clear the checkbox if you want Integration Server to use the value you specify only if the variable does not contain a value at run time.

Assigning String Variables to Pipeline Variables

In addition to assigning a literal value to a variable, you can assign the value of another String variable to a pipeline variable of type String, String List, or String Table. You might do this if you wanted to derive the default value from a variable that exists in the pipeline at run time.

To specify a String variable as the value for a pipeline variable, you enclose the name of the String variable in % symbols (for example, %Phone%) and then select the Perform pipeline variable substitution checkbox.

Keep the following information in mind when using pipeline variable substitution:

  • You can use String variables to specify values for pipeline variables of type String, String List, and String Table only.
  • You must select the Perform pipeline variable substitution checkbox for the variable substitution to occur.
  • You can mix literal values with String variables. For example, if you specify (%areaCode%) %Phone%, the resulting String would be formatted to include the parentheses and space. If you specify %firstName% %initial%. %lastName%, the period and spacing would be included in the value.
  • If the variable name that you specify between % does not exist in the pipeline at run time, no variable substitution occurs. Instead, the value of the pipeline variable will be %StringVariableName%. If you specify %lastName% as the value for the familyName variable, and the lastName variable does not exist in the pipeline at run time, the familyName variable value is: %lastName%
  • You can assign a pipeline variable a value comprised of a global variable and a pipeline variable. For more information, see Assigning String Variables and Global Variables to the Same Pipeline Variable.

Assigning Global Variables to Pipeline Variables

A global variable is a key/value pair that you define using Integration Server Administrator. You can use a global variable in a flow service by assigning the global variable to a variable in the pipeline (specifically a variable in Service In or Pipeline Out). At run time, Integration Server uses the value of the global variable as the value of the pipeline variable.

Using global variables makes it easy to change the value assigned to a pipeline variable at run time or after you deploy a solution to a different Integration Server. Instead of changing the hard-coded value of a pipeline variable, you change the value of the global variable. For example, you could create global variables for the connection parameters required by pub.client:ftp service. You might create global variables for the FTP server name, a user on the FTP server, and password for that user. In flow services that invoke pub.client:ftp you could assign the global variables to the serverhost, username, and password input parameters. After deploying the flow service to different servers, you may need to use different values for the serverhost, username, and password input parameters. You can use Integration Server Administrator to change the value of the global variables on the target servers. This is more efficient than editing the services to change the hard-coded values.

You can define global variables at the server level or package level. All services in Integration Server have access to a server-level global variable, whereas a package-level global variable is only accessible within the same package as the service. For more information about defining global variables, see webMethods Integration Server Administrator’s Guide.

When Integration Server encounters global variable substitution during service execution, Integration Server first looks for the global variable in the same package as the service. If not found, Integration Server looks for the global variable at the server level. You can override this default behavior by specifying whether the service must use a server-level or a package-level global variable. To use a server-level global variable, prefix the variable name like this: %server:variablename%. To use a package-level global variable, prefix the variable name like this: %package:variablename%.

Keep the following points in mind when using global variables in flow services:

  • To specify a global variable as the value for a pipeline variable, you enclose the name of the global variable in % symbols (for example, %myFTPServer%).
  • You can use global variables to specify values for variables of type String, String List, and String Table only. For more information, see Assigning a Value to a Pipeline Variable.
  • You can mix literal and global variables. For example, if you specify (%areaCode%) %Phone%, the resulting String would be formatted to include the parentheses and space. If you specify %firstName% %initial%. %lastName%, the period and spacing would be included in the value.
  • You can assign a pipeline variable a value comprised of a global variable and a pipeline variable. For more information, see Assigning String Variables and Global Variables to the Same Pipeline Variable.
  • If global variables at the server and package level are identical, use the server: or package: prefix to indicate the global variable that you want to use.
  • You must select the Perform global variable substitution checkbox for the variable substitution to occur at run time.
  • If the specified global variable has the same name as a pipeline variable name and you select the Perform global variable substitution checkbox and the Perform pipeline variable substitution checkbox, Integration Server uses the value of the pipeline variable at runtime.
  • If the global variable that you specified for performing a variable substitution is not defined in Integration Server, at runtime Integration Server throws an exception and service execution fails.

For more information about defining global variables, see webMethods Integration Server Administrator’s Guide.

Assigning String Variables and Global Variables to the Same Pipeline Variable

You can use global variable substitution and pipeline variable substitution for the same pipeline variable. That is, you can assign a pipeline variable of type String and a global variable to the same pipeline variable of type String, String List, or String Table. For example, you might assign a pipeline variable the following value: %myGlobalVariable% %myString% where myGlobalVariable is a global variable defined on Integration Server and myString is a variable that exists in the pipeline.

Keep the following information in mind when performing global variable substitution and pipeline variable substitution for the same variable.

  • To instruct Integration Server to perform global variable substitution and pipeline variable substitution, you must select the Perform global variable substitution checkbox and the Perform pipeline variable substitution checkbox.
  • If the specified global variable has the same name as a pipeline variable name and you select the Perform global variable substitution checkbox and the Perform pipeline variable substitution checkbox, Integration Server uses the value of the pipeline variable at run time.
  • If the pipeline variable that you specified does not exist in the pipeline at run time, Integration Server looks for a global variable with that name. If a global variable with that name is not defined in Integration Server, at run time Integration Server throws an exception and service execution fails

    For example, suppose that you specify %myGlobalVariable% %myString% where myGlobalVariable is a global variable defined on Integration Server and myString is a variable that exists in the pipeline. You also select the Perform global variable substitution checkbox and the Perform pipeline variable substitution checkbox. At run time, if Integration Server cannot find a String pipeline variable named myString, Integration Server looks for a global variable named myString. If a global variable named myString does not exist on that Integration Server then Integration Server throws an exception and service execution fails.

    If you anticipate that the pipeline variable that you want to use as the value pf another pipeline variable might not exist in the pipeline at run time, do not perform pipeline variable substitution an global variable substitution for that variable in the same flow step.

Copying Assigned Values Between Pipeline Variables

About this task

You can copy the value assigned to a variable by copying the icon next to the variable. You can assign this value to other variables of the same data type in Service In or Pipeline Out by pasting the icon.

When you copy assigned values from one pipeline variable to another, keep the following points in mind:

  • You can only copy and paste set values between variables of the same data type. For example, you can only copy the set value assigned to a String variable to another String variable.
  • You can only copy and paste set values between variables if the target variable has the same structure as the source variable or has no defined structure. For example, you can copy the set value of a String List variable with length 3 to another String List variable only if the target String List also has length 3 or has an undefined length (no defined structure).
  • If you are copying a set value between Document variables, the source Document variable and the target Document variable must have the same structure or the target Document variable must have no structure defined. For example, if the source Document variable contains three String variables named city, state, and zip as children, the target Document variable must have three String variables named city, state, and zip as children.
  • You cannot copy an assigned value to a pipeline variable if the variable has a fixed null or default value assigned to it. Designer uses the symbol next to the variable icon to indicate that the variable has a fixed value that you cannot override by linking it to another variable.

To copy a set value

Procedure

  1. In the flow service editor, select the INVOKE or MAP step containing the variable with the value you want to copy and paste.
  2. In the Pipeline view, select the assigned value icon that you want to copy.
  3. Right-click and select Copy.
  4. Select the variable to which you want to assign the copied value, right-click and select Paste.

Dropping Variables from the Pipeline

About this task

You can remove a variable from Pipeline In or Pipeline Out by dropping the variable. You can drop variables to eliminate pipeline variables that are not used by subsequent services in a flow. Dropping unneeded variables reduces the size of the pipeline at run time. At design time, dropping unneeded variables reduces the length and complexity of the Pipeline In and Pipeline Out displays. This can make the Pipeline view much easier to use when you are working with a complex flow.
Important: Software AG recommends that you drop variables containing sensitive data from the pipeline once the data is no longer needed by the flow service. Sensitive data includes, but is not limited to, passwords, private keys, and pass phrases. The pipeline stores data in clear text. Because the entire output pipeline is returned to the calling client, avoid leaving variables containing sensitive data in the pipeline. This prevents the service from returning sensitive data to the calling client.

Keep the following points in mind when dropping variables from the pipeline:

  • You can only drop variables from Pipeline In and Pipeline Out. In a MAP step, you can only drop variables from Pipeline In.
  • Once you drop a variable from the pipeline, it is no longer available to subsequent services in the flow. Do not drop a variable unless you are sure the variable is not used by services invoked after the point where you drop it.
  • At run time, Integration Server removes a dropped variable from the pipeline just before it executes the selected service (if you drop a variable in Pipeline In) or immediately after it executes the selected service (if you drop a variable in Pipeline Out).
  • If you drop a linked variable from Pipeline In, Integration Server executes the link before it drops the variable. However, Integration Server server does not link a null value to the destination variable.
  • You cannot drop a pipeline variable if the variable has a fixed null or default value assigned. Designer uses the symbol next to the variable icon to indicate that the variable has a fixed value that you cannot override by linking it to another variable.
  • You cannot drop a pipeline variable in a child flow service if the variable exists in the parent flow service. That is, a child flow service cannot drop an “upstream” variable.

To drop a variable from the pipeline

Procedure

  1. In the flow service editor, select the INVOKE or MAP step whose pipeline variables you want to drop.
  2. In the Pipeline view, select the variable that you want to drop.
  3. Click on the toolbar.

Adding Variables to the Pipeline

About this task

In the Pipeline view, you can add variables that were not declared as input or output parameters for the flow service itself or any of its constituent services. You can add variables that were omitted from a service’s input or output parameters or create temporary variables for use within the flow. For example, you might attach a variable to each of the children in a BRANCH step to mark the path taken by the service at run time.

Variables that you add to the pipeline can be used just like any declared variable in the flow.

Keep the following points in mind when adding variable to the pipeline:

  • If you create a new variable in a flow, you must immediately do one of the following:
    • Link a variable to it
    • Assign a value to it
    • Drop it

    If you do not take one of these steps, Designer automatically clears it from the pipeline.

  • You might want to drop a variable immediately after adding it if a service produces a variable that is not declared in the service input or output parameters. The variable will not appear in the Pipeline view if it is not an input or output parameter. By adding and then immediately dropping the variable, you can delete the variable if it does exist in the pipeline.

To add a variable to the pipeline

Procedure

  1. In the flow service editor, select the INVOKE or MAP step that represents the stage of the pipeline at which you want to add a new variable.
  2. Do one of the following in the Pipeline view:
    • Select the point in the pipeline where you want to add the new variable (Pipeline In, Service In, Service Out, or Pipeline Out). Click and select the type of variable that you want to create.
    • In the Palette view that is part of Pipeline view, under Variables, select the variable that you want to add and then select the point in the pipeline where you want to add it. The Palette view is located within the Pipeline view. Click to show the Palette view. Click to hide the Palette view.
  3. Type a name for the variable and press ENTER.
    Note: Do not use the % symbol in variable names. The % symbol is a reserved character because it is used in expressions for BRANCH steps and trigger filters.
  4. With the variable selected, set variable properties and apply constraints using the Properties view.
  5. If the variable is a Document or a Document List, add more variables to define its contents. Then use to indent each member variable beneath the Document or Document List variable.
  6. Do one of the following with the new variable:
    • Link the variable to another variable.
    • Assign a value to the variable using on the Pipeline view toolbar.
    • Drop the variable.

Using Smart Mapping

About this task

Designer provides the Smart Mapping functionality under the Pipeline view for MAP and INVOKE steps to simplify the mapping between the pipeline variables. You can use the Smart Mapping for flow services that provides the following capabilities.
  • Automatic prediction: Designer displays the predictions while mapping the pipeline data between input and output variables. You can use the suggested mappings to create a link between variables easily while working with a flow service.
  • Probability-based predictions: Designer displays the mapping predictions based on the likelihood of the input and output variables. This enables you to select the mappings effortlessly.
  • Tenant specific predictions: Designer connects to webMethods Cloud Container that uses a Machine Learning (ML) algorithm to suggest the predictions. This algorithm provides suggestions to map similar fields according to the mappings of the tenant. You can choose to connect to different cloud connections to obtain different mapping predictions.

Consider the following when you build a Smart Mapping:

  • Designer connects to the default Cloud Container connection to provide the mapping predictions. If this connection is inactive or does not support Smart Mapping, Designer does not show mapping predictions in the Pipeline view.
  • You may alternatively configure a Cloud Container connection under Window > Preferences > Cloud Container. To configure a Cloud Container, see Configuring a Connection to Cloud Container.

To use Smart Mapping

Procedure

  1. In the Package Navigator view, open the flow service.
  2. In the editor, select the INVOKE or MAP step for which you want to use Smart Mapping.
  3. In the Pipeline view, enable smart mapping by selecting the Mapping predictions checkbox.
    To hide the original mappings, select the Hide original mappings checkbox.
  4. The mapping prediction includes the following levels based on their mapping likelihood.
    • Very likely: Displays the highest probability.
    • Medium likely: Displays the medium probability.
    • Little likely: Displays the lowest probability.

    You can select a prediction and view the properties under the Properties window.

    Note: If no mappings are available for the pipeline input in Cloud Container, Designer does not display any mapping predictions.
  5. For the predictions under each category, you can perform the following actions:
    1. You can click Very likely, Medium likely, or Little likely mapping predictions to see the recommendations between the pipeline data.
    2. To accept a prediction, select and right-click the recommendation line, and click Select suggestion(s).

      Click Accept selected in the upper-right corner. You can alternatively click Accept All to select all the recommendations.

      Note: You can also undo the operation using the Edit > Undo command.
  6. When the mappings are completed, save all your changes using the File > Save All or Save command.

Working with Transformers

Transformers are services that are inserted into and executed within a MAP step. You can use any service as a transformer. This includes any Java, C, or flow service that you create and any built-in services in WmPublic, such as the pub.date.getCurrentDateString and pub.string.concat. Additionally, you can insert multiple transformers into a single MAP step. By using multiple transformers, you can perform multiple data transformations on the pipeline contents in a single flow step.

Transformers act as collection of INVOKE steps embedded in a single MAP step. However, transformers in a MAP step are independent of each other, do not execute in a specific order, and might not execute in the same order each time the MAP step runs. Consequently, the output of one transformer cannot be used as the input to another transformer. These characteristics make transformers different than a set of INVOKE steps that execute sequentially in a flow service. Because transformers are contained within a MAP step, they do not appear as a separate flow step in the editor.

The purpose of transformers is to accomplish multiple data transformations on the pipeline data in a single step as opposed to using succession of INVOKE steps. As a result, transformers are well suited to use when mapping data from one document format to another. When mapping data between formats, you often need to perform several name, structure, and value transformations. With the use of transformers, the flow service in which you map data between formats could potentially consist of a single MAP step in where transformers and links between variables handle all of the data transformations. This provides a single view of document-to-document mapping. For example, you could create a flow service that uses transformers to convert data between document formats (such as an IDOC to an XML document or RosettaNet PIP to a proprietary format). Each time you need to convert between the specific document formats, you could invoke the mapping service.

Note: When determining which services to use as transformers, Software AG recommends avoiding services that are subject to transient failures, such as a connection failure, as these services may be difficult to debug when used as a transformer.

Using Built-In Services as Transformers

Integration Server provides several built-in services specifically designed to translate values between formats. These services can be found in the following folders in the WmPublic package:

This folder… Contains services to…
pub.date Transform time and date information from one format to another.
pub.list Transform a String List to a Document List (IData[ ] object) and append items to a Document List (IData[ ] object) or a String List.
pub.math Perform simple arithmetic calculations (add, subtract, multiply, and divide) on integers and decimals contained in String variables.
pub.string Transform String values in various ways (for example, pad, substring, concat, replace through a lookup table).

For more information about built-in services, see the webMethods Integration Server Built-In Services Reference.

Inserting a Transformer

About this task

When inserting transformers, keep the following points in mind:
  • Transformers can be inserted in a MAP step only.
  • Any service can be used as a transformer, including flow services, C services, and Java services.
  • The transformers in a single MAP step operate on the same set of pipeline data.
  • Transformers in a MAP step are independent of each other and do not execute in a specific order. As a result, the output of one transformer cannot be used as the input of another transformer in the same MAP step.
  • Software AG recommends avoiding the use of a service as a transformer if the service is subject to transient failures, such as a connection failures, as these services might be hard to debug when used as a transformer.

To insert a transformer

Procedure

  1. In the flow service editor, select the MAP step in which you want to insert a transformer.
  2. In the Pipeline view, do one of the following:
    • Click the button adjacent to on the Pipeline view toolbar and select the service you want to use as a transformer. If the service you want to insert does not appear in the list, click Browse to select a service on Integration Server.
    • In the Palette view that is located within the Pipeline view, select the folder containing the service you want to add as a transformer. Select the service and click in the Transformers area of Pipeline view.
    • In Package Navigator view, select the service you want to use as a transformer and drag it to the Transformers area of Pipeline view.
  3. To set properties for the transformer, select it and then specify the following information in the Properties view:
    For this property... Specify...
    Service The fully qualified name of the service that will be invoked at run time as a transformer. When you insert a transformer, Designer automatically assigns the name of that service to the service property. If you want to change the service that is invoked by a transformer, specify the service’s fully qualified name in the folderName:serviceName format or click to select a service from a list.
    Validate input Whether Integration Server validates the input to the transformer against the signature of the service. Select True to validate the input of the transformer, otherwise select False.
    Validate output Whether Integration Server validates the output of the transformer against the signature of the service. Select True to validate the output of the transformer, otherwise select False.
  4. Link pipeline variables to the transformer variables. See Linking Variables to a Transformer.

Linking Variables to a Transformer

About this task

When you map data to and from a transformer, you create links between the pipeline variables and the transformer. Keep the following points in mind when you create links between pipeline and transformer variables:
  • You must explicitly link pipeline variables to the input and output variables of a transformer. Designer does not perform any implicit linking with transformers. Even if the pipeline variables have the same name and data type as the transformer variables, no implicit linking occurs.
  • Designer does not automatically add the output of a transformer to the pipeline. If you want the output of a transformer to appear in the pipeline, you need to explicitly link the output variable to a Pipeline Out variable.
  • If you do not link any output variables or the transformer does not have any declared output variables, the transformer service will not run.
  • You can link a transformer output variable to more than one Pipeline Out variable.
  • You can assign a value to a transformer input value using on the Pipeline view toolbar.
  • To prevent the Pipeline view from becoming cluttered, the Pipeline view may not display all the links between the transformer and the pipeline variables. To view all the links, double-click the transformer or click next to the transformer name.

Use the following procedure to link pipeline and transformer variables when the transformer is not expanded. If the transformer is expanded (that is, you can see all of the input and output variables for the transformer), you link variables just as you would for an INVOKE step.

To create a link between a pipeline variable and a transformer

Procedure

  1. To create a link between a Pipeline In variable and a transformer variable, do the following:
    1. In Pipeline In, select the variable you want to use as input to the transformer and drag your mouse to the collapsed transformer. Designer displays the Link dialog box.
    2. In the Link To list, select the transformer variable to which you want to link the Pipeline In variable.

      In the Link Tolist, Designer displays the phrase “has already been chosen” next to variables that are already linked to other variables transformer.

  2. To create a link between a transformer output variable and a Pipeline Out variable, do the following:
    1. Select the collapsed transformer and drag your mouse to the variable in Pipeline Out to which you want to link the transformer variable. Designer displays the Link dialog box.
    2. In the Link From list, select the transformer variable that you want to link to the selected Pipeline Out variable.

Transformers and Array Variables

When creating links between pipeline variables and transformers, dimensional differences between the source and target variables may cause an exception. If the target variable dimensionality is greater than the source variable dimensionality, an exception will not be thrown. However, if the source variable dimensionality is greater than the target variable dimensionality, Integration Server throws an exception.

Dimensionality refers to the number of arrays to which a variable belongs. For example, the dimensionality of a single String is 0, that of a single String List or Document List is 1, and that of a single String Table is 2. A String that is a child of a Document List has a dimensionality of 1. A String List that is a child of a Document List has a dimensionality of 2.

Example of Dimensionality Mismatch

In the following example, the unitPrice variable cannot be linked to num1 because the unitPrice variable has a dimensionality of 1 (String (0) + Document List (1) = 1) and num1 has a dimension of 0.

Figure 14. unitPrice cannot be linked to num1 because of dimensionality differences

To solve this, you can either:

  • Change the service invoked by the transformer to accept arrays as data, or
  • Create a flow service in which a LOOP step loops over the array variable. Then, (in the same flow service) invoke the service you originally wanted to use as a transformer, and make that INVOKE step a child of the LOOP. Finally, insert the resulting flow service as a transformer in the MAP.

Of the two options, changing the service to accept arrays as data results in faster execution of flow services.

Validating Input and Output for Transformers

About this task

As with any service you insert using an INVOKE, you can validate the inputs and outputs of the transformer service before and/or after it executes. To indicate that you want to validate a transformer’s inputs and outputs, you change the properties of the transformer. You do not have to use validation for all of the transformers you insert into a MAP step.

When Integration Server validates a transformer’s inputs and outputs at run time, Integration Server validates the transformer input and output values against the signature of invoked service. Variables in the service signature may specify content or structural constraints.

Note: If the Validate input and/or Validate output checkboxes are selected on the Input/Output tab of the service acting as a transformer, Integration Server automatically validates the input and/or output for the service every time the service executes. If you set up validation via the properties for a transformer when it is already set up for validation via the service’s Input/Output tab, Integration Server performs validation twice. This can slow down the execution of a transformer and, ultimately, the flow service.

To specify input/output validation for a transformer

Procedure

  1. In the flow service editor, select the MAP step containing the transformer you want to validate.
  2. In the Pipeline view, under Transformers, select the transformer for which you want to specifying input/output validation.
  3. In the Properties view, do the following:
    • If you want Integration Server to perform input validation, set the Validate input property to True.
    • If you want Integration Server to perform output validation, set the Validate output property to True.

Copying Transformers

About this task

You may want to use the same transformer more than once in a MAP step. For example, you might want to convert all the dates in a purchase order to the same format. Instead of inserting the service repeatedly, you can copy and paste the transformer service.
  • You can copy transformers between MAP steps in the same flow or MAP steps in different flow services.
  • You can copy multiple transformers at a time.
  • Copying a transformer does not copy the links between transformer variables and pipeline variables or any values you might have assigned to transformer variables using .

To copy a transformer

Procedure

  1. In the flow service editor, select the MAP step containing the transformer service you want to copy.
  2. In the Pipeline view, under Transformers, select the transformer that you want to copy. Right-click and select Copy.
  3. Do one of the following:
    • To paste the transformer in the same MAP step, right- click anywhere under Transformers and select Paste.
    • To paste the transformer in another MAP step, select that MAP step. In Pipeline view, right- click anywhere under Transformers and select Paste.
  4. Link the input and output variables of the transformer. See Linking Variables to a Transformer.

Renaming Transformers

About this task

If Integration Server displays the message “Transformer not found” when you try to expand a transformer or when you point the mouse to the transformer, then the service referenced by the transformer has been renamed, moved, or deleted. You need to change the Service property of the transformer so that the transformer points to the moved, or renamed service.

If the service referenced by the transformer has been deleted, you may want to delete the transformer.

Tip: You can enable safeguards so that you do not inadvertently affect or break other services when you move, rename, or delete a service. For more information, see Configuring Dependency Checking for Elements.

To rename a transformer

Procedure

  1. Use Package Navigator view to determine the new name or location of the service called by the transformer.
  2. Open the flow service containing the transformer you want to rename.
  3. In the flow service editor, select the MAP step containing the transformer. Then, in Pipeline view, select the transformer you want to rename.
  4. In the Service property in the Properties view, delete the old name and type in the service’s new fully qualified name in the folderName:serviceName format, or click to select a service from a list.

Debugging Transformers

When you debug a flow service, you can use the following debugging techniques with transformers:

Testing Flow Steps Before Running a Flow Service

As you construct a flow service, you can separately test the INVOKE or MAP steps that you plan to use in the flow service. For each step, you can provide relevant inputs, run the step, and check the output before running the entire flow service. This approach helps in early validation of a MAP or INVOKE step before it passes data to the next step in the flow.

You need to use the Data Mapper view in Designer for testing the flow steps.

Opening the Data Mapper View

About this task

To open the Data Mapper View

Procedure

  1. In the Service Development perspective of Designer, select Window > Show View > Other.
  2. In the Show View dialog box, select Software AG Service Development > Data Mapper.
  3. Click OK.
    Designer displays the Data Mapper view.

Testing a Flow Step in the Data Mapper View

About this task

To test a flow step in the Data Mapper View

Procedure

  1. In the Package Navigator view, open the flow service for which you want to insert the required INVOKE or MAP steps.
  2. Add the required flow steps, as explained in Inserting Flow Steps.
  3. Select the flow step to test, and switch to the Data Mapper view.
  4. In the Mapping tab of the Data Mapper view, define the required pipeline variables and transformers, and create links.
    You can perform these operations in the Mapping tab as you would in the Pipeline view. For more information, see the following sections:
  5. In the Testing tab of the Data Mapper view, specify values for the input variables of the flow step in the Input Value Creation area.
    You can use either of the following approaches to specify the values:
    • In the Value column of the Input Value Creation area, type the required values against the corresponding variable names listed in the Name column.
    • To load input values, which match the structure of the flow service's input signature, from a file; click Load, and select the appropriate file.
    • To load input values from a file, and replace the flow service's input signature with structure and data types from the file; click Load and Replace, and select the appropriate file.
    Note: For more information about loading input values from a file, see Loading Input Values.
  6. If you want to save the specified input values for later use, click Save.
    Otherwise, directly go to Step 7.
  7. To test the flow step based on the specified input values, click .
    Designer displays the results of the test in the Test Outcome area of the Data Mapper view.
    Depending on the results of a test execution, the Test Outcome area displays information in the following tabs:
    • The Pipeline tab displays the contents of the pipeline when the test finishes executing.
    • The Messages tab displays messages from Designer about the test and any exception thrown during the execution.

Mapping Using ForEach

To simplify array mapping, Designer provides ForEach mapping in the pipeline view of a MAP step. ForEach mapping provides following capabilities:

  • Reduces multiple flow steps: Nested array mapping involves multiple flow steps. Now, using ForEach mapping, you can easily create nested mapping in a single MAP step.
  • Copy modes: When an input array is mapped to an output array, the value of output array is merged with input array. Using ForEach mapping, you can now merge, overwrite, and append the values to output array.
  • Multiple transformer invocation: ForEach mapping allows you to directly apply multiple transformers on input array elements while mapping.

Creating a ForEach Mapping

About this task

Consider the following when you build a ForEach mapping:

  • You can only map an input array to an output array of the same data type.
  • ForEach mapping iterates over an input array and executes the defined mappings.
  • You can perform all the mapping tasks, transformations, and create nested array mapping in a ForEach mapping.
  • You can map a single input array to multiple output arrays and multiple input arrays to a single output array.

To build a ForEach mapping

Procedure

  1. In the flow service editor, select the MAP step.
  2. Open the Pipeline view.
  3. In Pipeline In, click the pipeline array variable you want to use as the source variable and in Pipeline Out, click the pipeline array variable you want to use as the target variable.
  4. Click .
    The ForEach mapping is established between the source and target arrays.
  5. On the Pipeline view of the ForEach mapping, click .
    The Pipeline view of the selected ForEach mapping appears.
  6. In the Pipeline view ForEach mapping, add the required mapping tasks, transformations, or nested mapping.
  7. Use to go to the Pipeline view of the parent ForEach, in case of nested ForEach mapping.
  8. Use to close the ForEach mapping Pipeline view.
  9. Click File > Save .

Specifying ForEach Mapping Properties

About this task

Following are the properties of ForEach mapping:

  • Filter Input: Specifies the matching condition for filtering the input array elements. At run time, Integration Server evaluates the condition for each input array element and if the condition evaluates to true, the mappings defined in the ForEach mapping pipeline are executed.
    Note: For information about the syntax used in conditions, see Conditional Expressions.
  • Copy Mode: Specifies how to copy the elements from input to output array. Following copy modes are supported:
    • Append: Appends the output array.
    • Merge: Updates the existing output array with source values. This is the default mode.
    • Overwrite: Overwrites the output array.
  • Elements Selection: Specifies the indexes of the input array that participate in the ForEach mapping. The first index element is 0. You can specify multiple cardinalities using the " , " delimiter. Consider the following example:
    If you specify an index value... Then ForEach processes...
    3 only the fourth element of the input array.
    1,5 only the second and the sixth elements of the input array.
    1-6,9,10 all second to seventh, tenth, and eleventh elements of the input array.
    Empty all the elements of the input array.
    Note: You should specify the indexes in the increasing order.
Note: When you specify both Elements Selection and Filter Input, then ForEach first selects elements based on the Elements Selection and then applies the Filter Input conditions on selected elements.

To specify ForEach mapping properties

Procedure

  1. Create a ForEach mapping between the array variables. See Creating a ForEach Mapping.
  2. On the ForEach mapping, click .
    The ForEach Settings dialog box appears.
  3. On the Filter Input screen, enter the filter condition for the ForEach mapping and click Next.
  4. In the Copy Mode screen, select the required copy action and click Next.
  5. In the Elements Selection screen, enter the indexes of the input array.
  6. Click Finish.
    A ForEach mapping is established between the selected variables.

ForEach Mapping Rules

The following tables identify the default ForEach mapping rules for array variables.

The example below illustrates a ForEach mapping between Employee (input array) and Person (output array).

  1. In aForEach mapping, you cannot link the individual array elements in a Pipeline view. To do this, you must expand the ForEachmapping. Also, you can build aForEach mapping from a nested input array, only if parent array is part of ForEach mapping.
    Following table identifies the possible cases in Pipeline view.
    From... To... Then…
    /Employee/id /Person/name This is an invalid link.

    Employee and Person arrays are already part of ForEach mapping. To create this link, go to the Pipeline view of the ForEach mapping.

    Age Person/age This is an invalid link.

    Person is part of current ForEach mapping but Age is not.

    To create this link, go to the Pipeline view of the ForEach mapping.

    Employee Location This is a valid ForEach mapping.

    Single input array (Employee) to multiple output array (Location and Person).

    CTC Person This is a valid ForEach mapping.

    Multiple input arrays (CTC and Employee) to single output array (Person).

    Employee Products This is an invalid ForEach mapping.

    Employee and Products are of different data types.

    Employee/address Person/city This is an invalid ForEach mapping.

    Employee and Person are already part of current ForEach mapping.

    To establish this nested ForEach mapping, go to the Pipeline view of the ForEach mapping.

    RnD/products Products This is an invalid ForEach mapping.

    RnD is not part of any ForEach mapping.

    To establish this nested ForEach mapping, add a ForEach mapping for RnD in the Pipeline view and then go to the Pipeline view of the ForEach mapping.

  2. In the Pipeline view of the ForEach mapping, a link is allowed only if the source variable, target variable, or both the variables are part of the current ForEach mapping array elements.
    Following table identifies the possible cases in the Pipeline view of the ForEach mapping.
    From... To... Then…
    /Employee/id Age This is a valid link.

    Employee/id is part of the current ForEach mapping.

    /Employee/id Person/name This is a valid link.

    Employee and Person arrays are part of ForEach mapping.

    Experience Person/age This is a valid link.

    Person/age is part of ForEach mapping.

    RnD/products Products This is an invalid link.

    RnD is not part of parentForEach mapping (Employee>Person).

    To create this link, go back to Pipeline view of MAP step.

  3. In the Pipeline view of the ForEach mapping, you can build a nested ForEach mapping only from those input array elements that are part of the parent ForEach mapping.
    Following table identifies the possible cases:
    From... To... Then…
    /Employee/address Person/city This is a valid nested ForEach mapping.

    Employee/address is an element of parent ForEach mapping.

    /Employee/address Location This is a valid nested ForEach mapping.

    Employee/address is an element of parent ForEach mapping.

    RnD Department This is an invalid nested ForEach mapping.

    RnD is not an element of parent ForEach mapping (Employee).