I've got a question concerning some global variables in DS. My aim is to:
1. Get a value from DB or flat file
2. Use the value in WHERE clause of SELECT statement as a row filter
3. Replace the value with a new value derived from the query above (as MAX(col1)) and store it on disk or DB
This is typical "incremental extraction" scenario: assuming a source table has a field with constantly growing value (id or modification date) then all one has to do on frequent extraction is to get the increment (i.e. rows with the field's value greater than the field's greatest value from preceding extraction).
My question is: how to accomplish tasks 1 through 3 in DataStage?
Internal CDC is out of the question.