Data elements

Each column definition in a server job stage might have a data element assigned to it. A data element specifies the type of data a column contains, which in turn determines the transforms that can be applied in a Transformer stage.

The use of data elements is optional. You do not have to assign a data element to a column, but it enables you to apply stricter data typing in your design. The extra effort of expended in defining and applying data elements can pay dividends in effort saved later on when you are debugging your design.

You can use any of the built-in data elements or you can create data elements specific to your application. Application-specific data elements allow you to describe the data in a particular column in more detail. The more information you supply about your data, the more the Designer client can help to define the processing needed in each Transformer stage. For example, if you have a column containing a numeric product code, you could assign it the built-in data element Number. There is a range of built-in transforms associated with this data element. However, all of these would be unsuitable, as it is unlikely that you would want to perform a calculation on a product code. In this case, you could create a new data element called Pcode.

Each data element has its own specific set of transforms which relate it to other data elements. When the data elements associated with the columns of a target table are not the same as the data elements of the source data, you must ensure that you have the transforms needed to convert the data as required. For each target column, you should have either a source column with the same data element, or a source column that you can convert to the required data element. For example, suppose that the target table requires a product code using the data element Pcode, but the source table holds product data using an older product numbering scheme. In this case, you could create a separate data element for old-format product codes called Old_Pcode, and then create a custom transform to link the two data elements. Its source data element is Old_Pcode, while its target data element is Pcode. This transform would convert an old product code to a new product code. A data element can also be used to "stamp" a column with SQL properties when you manually enter a table definition or enter a column definition for a link in a job. Data element definitions are viewed or created in the Data Element window.