Configuring normal lookup operations
You can configure a Greenplum Connector stage to retrieve a set of records from a Greenplum database and provide them to a Lookup stage in the same job. The Lookup stage then performs a normal (in-memory) lookup operation on those records.
Before you begin
- Set up the column definitions on a link to specify the format of the records that the Greenplum Connector stage reads from a Greenplum server.
- Configure the Greenplum Connector stage as a source for the reference data.
- Add a Lookup stage to the job design canvas, and then create a reference link from the Greenplum Connector stage to the Lookup stage.
About this task
In a normal lookup, the connector runs the specified SELECT statement only one time; therefore, the SELECT statement cannot include any input parameters. The Lookup stage searches the result set data that is provided by the connector and looks for matches for the parameter sets that arrive in the form of records on the input link to the Lookup stage. A normal lookup is also known as an in-memory lookup because the lookup is performed on the cached data in memory. You can use a normal lookup when the target table is small enough that all of the rows in the table can fit in memory.