Talend Context Configuration

There are a few ways to load values into context variables which influence the way the Talend scanner processes them. User configuration is needed for the scanner to work properly. The configuration needs to be done for each defined Talend connection.

Associated Source System Properties

Property name	Description	Example
talend.iterateContext	Enter whether all possible contexts should be analyzed	true false

If you’re using iterateContext=true and your jobs have a lot of different context groups set up, memory usage can grow significantly. The general rule is that if the average number of contexts per job is for example 5, memory usage can, at certain points of dataflow analysis, grow up to 5 times. See Manta Flow Memory Settings for information on how to allocate additional resources to support the Talend contexts feature.

Associated Common Properties

Property name	Description	Example
talend.input.contextDir	Directory with Talend context files	${manta.dir.input}/talend/${talend.system.id}/context
talend.input.contextReplacement	File with manual context values	${manta.dir.input}/talend/${talend.system.id}/contextReplacement.txt

Context Logic

The context is loaded in three stages.

Static load
- Loads only the *.item file; this file contains the values that were directly set in Talend Studio
Implicit tContextLoad
- Loads the context file specified by the path
Replacement context load
- Loads manual values

This means that if a variable value is defined in both the static and implicit contexts, the implicit context value is used. The replacement context load happens last, so it overrides any previously found values for known variables.

Input Folder Structure

In the connection input folder, you should create a context folder, which will contain all the context files used in Talend jobs.
- Path to the context folder is stored in talend.input.contextDir
- Default value: ${manta.dir.input}/talend/${talend.system.id}/context
In the connection input folder, you can also create a contextReplacement text file.
- Path to the contextReplacement file is stored in talend.input.contextReplacement
- Default value: ${manta.dir.input}/talend/${talend.system.id}/contextReplacement.txt

Finally, the folder could look like this:

No alt text provided

Implicit tContextLoad

If a Talend job uses an implicit tContextLoad, follow the instructions below depending on how the context variables are loaded.

a. From a file

The file that you should place is an implicit context, which is a specific file that you set in the Talend project configuration. You can find the location of this file under Project Settings > Job settings > Implicit context load. If you use the "From File" option, there is a field leading to the ImplicitContext file placed within the /inputs/talend/{connectionName}/context/ folder. An exemplary implict context file can look as follows:

<elementParameter field="FILE" name="IMPLICIT_TCONTEXTLOAD_FILE" value="&quot;{path to the context}/contexts/onlyImplicit.txt&quot;"/> <elementParameter field="TEXT" name="FIELDSEPARATOR" value="&quot;=&gt;&quot;"/>

The IMPLICIT_TCONTEXTLOAD_FILE contains the path to the context file. The FIELDSEPARATOR specifies => as the delimiter. The file should contain a key, value pairs separated by the delimiter. Place each new key and pair on a new line e.g.

first=>impl1
second=>impl2
path=>pathino

Place the file with the context variables as is in the context folder.
- IBM Automatic Data Lineage then searches for the context file as follows. The search stops on the first match.
  1. Try to search the specified path in the context folder.
  2. If not found, iteratively remove directories from the beginning of the path and search for the remaining path within the context folder.
  3. Try to find the file in the current directory.

b. From a database

Automatic Data Lineage does not connect to database to query for the actual values. The required values for context need to be in the contextReplacement.txt file. The structure of this file is shown below in the section entitled Manual Context.

Manual Context

If you want to edit context values manually, use the contextReplacement.txt file. This file overrides all previously-loaded values so you can set up values manually or add new ones.

Structure of contextReplacement.txt

[jobName/testing]
key=value
key2=value2

[jobName/production]
key=value5
key2=value

[jobName2/testing]
key=value
key2=value2
key3=58

Each block of context values begins with [jobName/contextName], followed by any number of variableName=value rows. At the end of a block, there needs to be an empty line.

Default Behavior

If both the context folder ${talend.input.contextDir} and the contextReplacement.txt file are not used, only a static load is performed. If there are no values for some of the variables in the context after this step, the analysis will replace the empty values with a context.variableName string.

No alt text provided