Topic
  • 3 replies
  • Latest Post - ‏2013-03-21T16:04:28Z by SystemAdmin
KhandekarAkram
KhandekarAkram
14 Posts

Pinned topic How to push annotated data using jaql to a file

‏2013-03-19T12:52:50Z |
I want to use the results after using systemT to annotate a document using the .aog file (running extractor using jaql systemT module). How can I store the annotated data in a file (preferbly json)?

Regards,
Akram.
Updated on 2013-03-21T16:04:28Z at 2013-03-21T16:04:28Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    603 Posts

    Re: How to push annotated data using jaql to a file

    ‏2013-03-20T12:29:29Z  
    This sample of code is doing that, is BI 2.0, but I supposed the json recored generated by the annotator would be similar.
    import systemT;
    //inputfile = read(del(location="/user/HistoryInput/BigHistoryJson.txt", delimiter='|', schema = schema{ label,text}));
    inputfile = read(del(location="file:///home/biadmin/History/BigHistoryJson.txt", delimiter='\t', schema = schema{ label,text}));

    //annotate file documents with AQL annotators
    annotatedtext= array[];
    annotatedtext = inputfile -> transform { label: $.label, annotations:
    systemT::annotateDocument( $, ,
    , spanOutputPolicy="toJsonString", tokenizer="multilingual",
    outputViews=)};
    filteredtext = annotatedtext ->transform {label: $.label, precio:$.annotations."ProductPrices.Precio"[*].precio,producto:$.annotations."ProductPrices.ProducoInmobiliario"[*].nombreproducto};

    filteredtext ->write(seq(location="file:///home/biadmin/HistoryOut/BigHistoryJson.out",
    outoptions={format: "org.apache.hadoop.mapred.TextOutputFormat", converter: "com.ibm.jaql.io.hadoop.converter.ToJsonTextConverter"}));
  • KhandekarAkram
    KhandekarAkram
    14 Posts

    Re: How to push annotated data using jaql to a file

    ‏2013-03-20T16:50:38Z  
    This sample of code is doing that, is BI 2.0, but I supposed the json recored generated by the annotator would be similar.
    import systemT;
    //inputfile = read(del(location="/user/HistoryInput/BigHistoryJson.txt", delimiter='|', schema = schema{ label,text}));
    inputfile = read(del(location="file:///home/biadmin/History/BigHistoryJson.txt", delimiter='\t', schema = schema{ label,text}));

    //annotate file documents with AQL annotators
    annotatedtext= array[];
    annotatedtext = inputfile -> transform { label: $.label, annotations:
    systemT::annotateDocument( $, ,
    , spanOutputPolicy="toJsonString", tokenizer="multilingual",
    outputViews=)};
    filteredtext = annotatedtext ->transform {label: $.label, precio:$.annotations."ProductPrices.Precio"[*].precio,producto:$.annotations."ProductPrices.ProducoInmobiliario"[*].nombreproducto};

    filteredtext ->write(seq(location="file:///home/biadmin/HistoryOut/BigHistoryJson.out",
    outoptions={format: "org.apache.hadoop.mapred.TextOutputFormat", converter: "com.ibm.jaql.io.hadoop.converter.ToJsonTextConverter"}));
    I am currently using BigInsights 1.4; can you help me out with some suggestions?
  • SystemAdmin
    SystemAdmin
    603 Posts

    Re: How to push annotated data using jaql to a file

    ‏2013-03-21T16:04:28Z  
    I am currently using BigInsights 1.4; can you help me out with some suggestions?
    I suppose is the same.

    Just do:

    annotatedtext = call sysmteT annotators

    annotatedtext ->write(seq(location="/user/HistoryOutput/BigHistoryJsonOut",
    outoptions={format: "org.apache.hadoop.mapred.TextOutputFormat", converter: "com.ibm.jaql.io.hadoop.converter.ToJsonTextConverter"}));