Pinned topic QS Job with Reference Match and IST I/O stages: Issues and Questions
I am trying to design a quality stage job which will take
1. A single record Input as simple as the First Name, Middle Name and Last Name columns
2. Should be able to fetch the exact/duplicate matches from a DB table with the above columns based on NAME_UNCERT algorithm in Reference Match Stage.
3. This should be deployed as a web service with ISD Input and ISD output.
In cases like above scenario, I came to know that Look up stage must be used(http://pic.dhe.ibm.com/infocenter/iisinfsv/v8r7/index.jsp?topic=%2Fcom.ibm.swg.im.iis.infoservdir.user.doc%2Ftopics%2Ft_create_realtime_ref_match.html).
Here I have the following questions.
1. To use a look up stage I need to use a join column. Look up stage in my case is totally unnecessary. I don't have join key column to link in DB. I need to use ALL the records from reference DB for the Reference Match stage.
2. Assume that there are a million records in database. Can I use Look up stage to feed in all the million records to Reference Match Stage.
3. Just wondering why is this look up stage made mandatory here. In my scenario or any other scenario do we really need it ?
RobertDickson 0600009JMM69 Posts
Re: QS Job with Reference Match and IST I/O stages: Issues and Questions2013-07-14T18:22:03ZThis is the accepted answer. This is the accepted answer.
A lookup stage OR a Join (Use the OBDC stagewith the SPARSE option set) will work. Your key would be all records that meet your blocking strategy. For example, if you have one pass that is blocking on NYSIIS of last name and First Character of Match First Name, then those two columns would be your key.