Topic
  • 17 replies
  • Latest Post - ‏2014-03-03T19:02:56Z by malc1288
ravindra_b
ravindra_b
2 Posts

Pinned topic Need help on Classification file.

‏2012-05-09T16:25:20Z |
I am creating a ruleset and am trying to include the following forward in the classification file.
"SAN JOSE" SANJOSE W

However, the output is not as expected. Can any one help me out in figuring out the right way please.
Updated on 2012-05-10T18:56:37Z at 2012-05-10T18:56:37Z by smithha
  • smithha
    smithha
    23 Posts

    Re: Need help on Classification file.

    ‏2012-05-09T19:48:37Z  
    If you have a look at the documentation regarding the Classification file http://publib.boulder.ibm.com/infocenter/iisinfsv/v8r7/topic/com.ibm.swg.im.iis.qs.ug.doc/topics/c_Classification_table.html
    you'll find the following note regarding the token value: "One or more characters, excluding the blank character"

    Since in most cases you will not want to strip out blanks/spaces (as that will strip out all such characters), you have to treat "SAN" and "JOSE" as separate and distinct tokens in the Classification file and code accordingly in your Pattern Action language.

    For instance you could have the following in the Classification table:
    SAN SAN W
    JOSE JOSE W

    You could then have a pattern such as the following in the PAT file:
    W | W
    and then code the actions you want to take such as concatenating the two values without a space.

    Harald
  • ravindra_b
    ravindra_b
    2 Posts

    Re: Need help on Classification file.

    ‏2012-05-10T17:29:00Z  
    • smithha
    • ‏2012-05-09T19:48:37Z
    If you have a look at the documentation regarding the Classification file http://publib.boulder.ibm.com/infocenter/iisinfsv/v8r7/topic/com.ibm.swg.im.iis.qs.ug.doc/topics/c_Classification_table.html
    you'll find the following note regarding the token value: "One or more characters, excluding the blank character"

    Since in most cases you will not want to strip out blanks/spaces (as that will strip out all such characters), you have to treat "SAN" and "JOSE" as separate and distinct tokens in the Classification file and code accordingly in your Pattern Action language.

    For instance you could have the following in the Classification table:
    SAN SAN W
    JOSE JOSE W

    You could then have a pattern such as the following in the PAT file:
    W | W
    and then code the actions you want to take such as concatenating the two values without a space.

    Harald
    thanks much for the reply.

    I have the data in this format,

    TEST SAN JOSE xyz 1234

    and i was trying to figure out a way to parse that SAN JOSE and retain "TEST xyz 1234". If i have a single word instead of SAN JOSE it works.

    the pattern i am using is :

    *& =a= "SAN JOSE"
    ---

    • |
    copy_s [1] {test}
  • smithha
    smithha
    23 Posts

    Re: Need help on Classification file.

    ‏2012-05-10T18:56:37Z  
    thanks much for the reply.

    I have the data in this format,

    TEST SAN JOSE xyz 1234

    and i was trying to figure out a way to parse that SAN JOSE and retain "TEST xyz 1234". If i have a single word instead of SAN JOSE it works.

    the pattern i am using is :

    *& =a= "SAN JOSE"
    ---

    • |
    copy_s [1] {test}
    I'll suggest having a look at the existing COUNTRY.PAT file for a number of possible pattern examples. There's not necessarily one right way to approach this, and a lot depends on other data content.

    I've taken the following out of the COUNTRY.PAT file with some slight modification:

    This case would evaluate trailing position words if you just had TEST SAN JOSE without the xyz 1234 and did not want to formally code the word 'SAN':
    1. &="SAN" | W="JOSE" | $ ;

    This case would look for SAN JOSE followed by a word, followed by a 4 character numeric.
    1. &="SAN" | W="JOSE" | & | ^ {}LEN=4 $

    You could just put together a basic pattern like this starting with one or more words, followed by two classified values, then one word and one numeric token.
    ? | W | W | & | ^ $

    Again, a lot of different approaches can be used to parse this and the delivered pattern files provide a view into common approaches.

    Harald
  • malc1288
    malc1288
    8 Posts

    Re: Need help on Classification file.

    ‏2014-01-20T17:03:41Z  

    Hi, can you hep me with the next topic?... i want to standardize the next names:

    migu

    mig

    these are miguel..

    i have the next code:

    in classifications file

    ;-------

    ;    F    -    First Name

    ;-------

     

    migu miguel F

    mig miguel F

    in dictionary file:

    FirstName                C    10   S FirstName                      ;0001-0009

     

    in patterns file:

    ? | F

    copy_s [1] {test}

     

  • RobertDickson
    RobertDickson
    42 Posts

    Re: Need help on Classification file.

    ‏2014-01-20T21:55:09Z  
    • malc1288
    • ‏2014-01-20T17:03:41Z

    Hi, can you hep me with the next topic?... i want to standardize the next names:

    migu

    mig

    these are miguel..

    i have the next code:

    in classifications file

    ;-------

    ;    F    -    First Name

    ;-------

     

    migu miguel F

    mig miguel F

    in dictionary file:

    FirstName                C    10   S FirstName                      ;0001-0009

     

    in patterns file:

    ? | F

    copy_s [1] {test}

     

    Hi,

    A couple of things:

    1) What does your full input look like?  ? | F will not match an input name of "migu"

    2) If you are using the USNAME rule set, there is a lookup table in the ruleset called USFIRSTN.  The lookup table is where the first names (and genders) are converted.  The format of the table is:

    input_name<space><gender><standardized_name>

    So Bob gets a matchfirstname of Robert and a gender of M with the entry of: BOB MROBERT

    For you, you would need to add MIGU MMIGUEL

    I hope this helps.

    Regards,

    Robert

  • malc1288
    malc1288
    8 Posts

    Re: Need help on Classification file.

    ‏2014-01-20T22:52:46Z  

    Hi,

    A couple of things:

    1) What does your full input look like?  ? | F will not match an input name of "migu"

    2) If you are using the USNAME rule set, there is a lookup table in the ruleset called USFIRSTN.  The lookup table is where the first names (and genders) are converted.  The format of the table is:

    input_name<space><gender><standardized_name>

    So Bob gets a matchfirstname of Robert and a gender of M with the entry of: BOB MROBERT

    For you, you would need to add MIGU MMIGUEL

    I hope this helps.

    Regards,

    Robert

    thanks to reply,

    the full input are the next

    id- name

    1-migu

    2-mig

    3-miguel

    i want to standardize all names to MIGUEL, i´ve created an 'rule set' because USNAME is 'read only', how can i agree the lookup table USFIRSTN?

     

  • RobertDickson
    RobertDickson
    42 Posts

    Re: Need help on Classification file.

    ‏2014-01-21T11:31:42Z  
    • malc1288
    • ‏2014-01-20T22:52:46Z

    thanks to reply,

    the full input are the next

    id- name

    1-migu

    2-mig

    3-miguel

    i want to standardize all names to MIGUEL, i´ve created an 'rule set' because USNAME is 'read only', how can i agree the lookup table USFIRSTN?

     

    Hi,

    Instead of trying to recreate the entire rule set, it would probably be better to create a copy and use the copy.  You can create a copy by right-clicking on the 'SET' in the repository and select 'create copy'.  You can then right-click and rename the copy to be what you want.  Please take a look at http://pic.dhe.ibm.com/infocenter/iisinfsv/v9r1/index.jsp?topic=%2Fcom.ibm.swg.im.iis.qs.ug.doc%2Ftopics%2Ft_copying_rule_set.html.

     

    Regards,

    Robert

  • malc1288
    malc1288
    8 Posts

    Re: Need help on Classification file.

    ‏2014-01-21T19:10:44Z  

    Hi,

    Instead of trying to recreate the entire rule set, it would probably be better to create a copy and use the copy.  You can create a copy by right-clicking on the 'SET' in the repository and select 'create copy'.  You can then right-click and rename the copy to be what you want.  Please take a look at http://pic.dhe.ibm.com/infocenter/iisinfsv/v9r1/index.jsp?topic=%2Fcom.ibm.swg.im.iis.qs.ug.doc%2Ftopics%2Ft_copying_rule_set.html.

     

    Regards,

    Robert

    thanks much for the reply,

    i've created the copy of USNAME rule set, and the three tables: USFIRSTN, USGENDER, USNAMEMF. In "paterns file" changed the names of tables to associate with the new names, ex: USFIRSTN to CopyOfUSFIRSTN, then in "ReferenceTables" appear the new names; problem is when run the job:

    Standardize 1,0: Table could not be found

    Standardize_5,0: The above error occurred in the following line of file .\RT_QS1962\V0S5\Controls\USFIRSTN.TBL

    Is there another step to do to associate the new tables with the rule set?

    Can you help me with that ..

  • RobertDickson
    RobertDickson
    42 Posts

    Re: Need help on Classification file.

    ‏2014-01-21T21:58:36Z  

    Hi,

    Depending on the version, you may need to edit the PAT file (via the GUI) and change all occurrences of USFIRSTN.TBL to CopyOfUSFIRSTN.TBL, etc.

     

    For all versions, yiou will need to right-click on the set and select 'Provision All'.

     

    Regards,

    Robert

  • malc1288
    malc1288
    8 Posts

    Re: Need help on Classification file.

    ‏2014-01-22T18:11:32Z  

    Hi,

    Depending on the version, you may need to edit the PAT file (via the GUI) and change all occurrences of USFIRSTN.TBL to CopyOfUSFIRSTN.TBL, etc.

     

    For all versions, yiou will need to right-click on the set and select 'Provision All'.

     

    Regards,

    Robert

    thanks,

    the job runs correctly, but now the problem is in outputs the value of all fields is 'null'

    i´ve added in CopyofUSFIRSTNAME and in classiffications file the names that i need

    in CopyofUSFIRSTNAME:

    MIG MMIGUEL

    MIGU MMIGUEL

     

    in classifications file:

    MIG MIGUEL F

    MIGU MIGUEL F (F is represented by FIRST_NAME)

    thanks.

  • malc1288
    malc1288
    8 Posts

    Re: Need help on Classification file.

    ‏2014-01-22T23:25:30Z  
    • malc1288
    • ‏2014-01-22T18:11:32Z

    thanks,

    the job runs correctly, but now the problem is in outputs the value of all fields is 'null'

    i´ve added in CopyofUSFIRSTNAME and in classiffications file the names that i need

    in CopyofUSFIRSTNAME:

    MIG MMIGUEL

    MIGU MMIGUEL

     

    in classifications file:

    MIG MIGUEL F

    MIGU MIGUEL F (F is represented by FIRST_NAME)

    thanks.

    Thanks a lot my job is running, i know every time you make changes you need to select "provision all"

     

  • RobertDickson
    RobertDickson
    42 Posts

    Re: Need help on Classification file.

    ‏2014-01-23T13:54:33Z  
    • malc1288
    • ‏2014-01-22T23:25:30Z

    Thanks a lot my job is running, i know every time you make changes you need to select "provision all"

     

    Hi,

    What is the InputPattern for your data?

    Regards,

    Robert

  • malc1288
    malc1288
    8 Posts

    Re: Need help on Classification file.

    ‏2014-01-23T21:01:14Z  

    Hi,

    What is the InputPattern for your data?

    Regards,

    Robert

    hi,

    InputPattern?

    is that?

    MIG MMIGUEL

    MIGU MMIGUEL

    or..?

  • RobertDickson
    RobertDickson
    42 Posts

    Re: Need help on Classification file.

    ‏2014-01-24T02:39:36Z  
    • malc1288
    • ‏2014-01-23T21:01:14Z

    hi,

    InputPattern?

    is that?

    MIG MMIGUEL

    MIGU MMIGUEL

    or..?

    Hi,

    If you look in the output generated by the Standardize stage, there will be a column that starts with 'InputPattern'.  What are the contents of that column?

    Regards,

    Robert

  • malc1288
    malc1288
    8 Posts

    Re: Need help on Classification file.

    ‏2014-02-03T19:56:00Z  

    Hi,

    If you look in the output generated by the Standardize stage, there will be a column that starts with 'InputPattern'.  What are the contents of that column?

    Regards,

    Robert

    sorry for the delay,column input pattern = F+

  • RobertDickson
    RobertDickson
    42 Posts

    Re: Need help on Classification file.

    ‏2014-02-04T03:39:58Z  
    • malc1288
    • ‏2014-02-03T19:56:00Z

    sorry for the delay,column input pattern = F+

    Everything looks like it should be working, except that F+ should ALWAYS be handled by the out of the box rule sets.  There is something else going on.  Please open a support ticket with IBM.

     

    Regards,

    Robert

  • malc1288
    malc1288
    8 Posts

    Re: Need help on Classification file.

    ‏2014-03-03T19:02:56Z  

    Everything looks like it should be working, except that F+ should ALWAYS be handled by the out of the box rule sets.  There is something else going on.  Please open a support ticket with IBM.

     

    Regards,

    Robert

    Hi, can you help me with that, what is the best way to migrate clob fields using datastage 8.1?