Topic
  • 8 replies
  • Latest Post - ‏2013-10-15T09:34:48Z by NSchulz
RajeshKumarMandal
RajeshKumarMandal
21 Posts

Pinned topic Customized parser

‏2013-10-06T08:03:08Z |

Hello,

In streams we have TEDA where cdr* with ASN.1 encoded data can be parsed and processed accordingly.

Is there any option to have our own parser to parse the data which is not encoded with ASN.1 format but some other encoding format ?

I was looking into InfoCenter (v3r1) -> Accelerating the value of big data analytics -> Analyzing telecommunications event data to monitor network quality -> Analyzing your product and services -> Customizing your environment -> Implementing a parser

where they have asked to first run sample parser and run the same to set up environment and later customize grammer.asn file as per our changes.

But is this only for ASN.1 encoded files ?

Do we have the option to implement our own parser in streams?

Thanks,

Rajesh

 

 

  • wbratton
    wbratton
    76 Posts

    Re: Customized parser

    ‏2013-10-07T15:47:05Z  

    Hi Rajesh,

    We will research your question and get back to you.

    Thanks

  • RajeshKumarMandal
    RajeshKumarMandal
    21 Posts

    Re: Customized parser

    ‏2013-10-07T16:50:59Z  
    • wbratton
    • ‏2013-10-07T15:47:05Z

    Hi Rajesh,

    We will research your question and get back to you.

    Thanks

    Hello William,

    Waiting for ur reply :)

    Thanks,

    Rajesh

  • wbratton
    wbratton
    76 Posts

    Re: Customized parser

    ‏2013-10-10T20:03:00Z  

    We are still researching your question.

  • NSchulz
    NSchulz
    22 Posts

    Re: Customized parser

    ‏2013-10-14T09:02:42Z  

    Hi Rajesh,

    let me apologize first that we weren't able to provide you with a

    good answer so far, but I have been on vacation for the last week

    and there was no one else from the TEDA team available to monitor

    this forum.

    The IC mainly focuses on ASN.1 parsers as we thought that this would

    be the main situation developers will find themselves in when using TEDA,

    but you are free to remove the given parser and exchange it with another

    custom operator implementing the parser functionality you need.

    I think the Operator development reference and the Operator model reference

    chapters will come in handy when starting the task.

     

    What exactly do you need to do and which programming language do you

    want to use?

     

    Kind regards,

    Norbert

  • RajeshKumarMandal
    RajeshKumarMandal
    21 Posts

    Re: Customized parser

    ‏2013-10-14T16:49:23Z  
    • NSchulz
    • ‏2013-10-14T09:02:42Z

    Hi Rajesh,

    let me apologize first that we weren't able to provide you with a

    good answer so far, but I have been on vacation for the last week

    and there was no one else from the TEDA team available to monitor

    this forum.

    The IC mainly focuses on ASN.1 parsers as we thought that this would

    be the main situation developers will find themselves in when using TEDA,

    but you are free to remove the given parser and exchange it with another

    custom operator implementing the parser functionality you need.

    I think the Operator development reference and the Operator model reference

    chapters will come in handy when starting the task.

     

    What exactly do you need to do and which programming language do you

    want to use?

     

    Kind regards,

    Norbert

    Hello Norbert,

    We have a input source from DataCast which sends binary OHDR data over TCP/IP socket.

    We have the rule to decode this OHDR BLOB data. The size of each individual parameter, size and description are available with us.

    Thanks,

    Rajesh

  • NSchulz
    NSchulz
    22 Posts

    Re: Customized parser

    ‏2013-10-15T08:38:41Z  

    Hello Norbert,

    We have a input source from DataCast which sends binary OHDR data over TCP/IP socket.

    We have the rule to decode this OHDR BLOB data. The size of each individual parameter, size and description are available with us.

    Thanks,

    Rajesh

    Hi Rajesh,

    this setup bears another problem: The Recovery concept of TEDA relies on

    data being delivered in files. Is there a chance to create another Streams job,

    which reads your data from the TCP/IP socket, produces files from it, and feeds

    those files to TEDA?

     

    Kind regards,

    Norbert

     

  • RajeshKumarMandal
    RajeshKumarMandal
    21 Posts

    Re: Customized parser

    ‏2013-10-15T08:54:37Z  
    • NSchulz
    • ‏2013-10-15T08:38:41Z

    Hi Rajesh,

    this setup bears another problem: The Recovery concept of TEDA relies on

    data being delivered in files. Is there a chance to create another Streams job,

    which reads your data from the TCP/IP socket, produces files from it, and feeds

    those files to TEDA?

     

    Kind regards,

    Norbert

     

    Hello Nobert,

    Yes, we can add a streams job to read from TCP/IP and write into output files into a directory from which TEDA can access the files.

    So once we have the files we can have TEDA processing later on.

    So excluding his part to read data from DataCast which sends binary OHDR BLOB data over TCP/IP and writing into output files later on we can use TEDA ?

    Thanks,

    Rajesh

  • NSchulz
    NSchulz
    22 Posts

    Re: Customized parser

    ‏2013-10-15T09:34:48Z  

    Hello Nobert,

    Yes, we can add a streams job to read from TCP/IP and write into output files into a directory from which TEDA can access the files.

    So once we have the files we can have TEDA processing later on.

    So excluding his part to read data from DataCast which sends binary OHDR BLOB data over TCP/IP and writing into output files later on we can use TEDA ?

    Thanks,

    Rajesh

    Hi Rajesh,

    as long as you provide a parser to read the blob data that

    should be no problem. As you now provide a parallel job

    you have the freedom to choose where to put the DataCast

    parser. If you decide to create decoded CVS text files the

    parser part within TEDA becomes a simple CVS reader, or

    you may decide to simply store the binary data in your parallel

    job and create a DataCast parser that resides in TEDA.

     

    Kind regards

    Norbert