Topic
  • 1 reply
  • Latest Post - ‏2013-06-27T18:29:01Z by lmlewis
sakus
sakus
5 Posts

Pinned topic Using MDAGenerateMetadata.sh for CSVs

‏2013-06-27T12:26:43Z |

Hi,

I have a challenge as I'm not able to generate .json file for my data batch. I'm using MDAGenerateMetadata.sh, but get following message "Warn: When logtype is CSV, please make sure all csv files have header."  when running the command ./MDAGenerateMeta.sh logtype="csv" path="/flow/csv" dateTimeFormat="MMM-dd-yyyy H:mm:ss" delimiter=","

There are only two .csv files in the folder and both have header lines stating the name of the columns. I have not found any good tutorials for using Machine Data Accelerator. Most of the example use predefined metadata.json files and do not cover the actual generation of the .json file.

Any help is appreciated.

Br,
Saku

  • lmlewis
    lmlewis
    1 Post

    Re: Using MDAGenerateMetadata.sh for CSVs

    ‏2013-06-27T18:29:01Z  

    Hi Saku,

    The MDAGenerateMetadata.sh script is documented in the BigInsights 2.0 Information Center under:

    There is also more information about the MDAGenerateMeta.sh script, including examples, in the MDAGenerateMeta_readme file in the MDA_install_location/bin directory.  Based on the samples in this readme and my own tests, this warning always appears then the logType is csv, but the metadata.json file is still created in the path where your .csv files reside

    If a metadata.json file is not getting generated, then perhaps there is something wrong with the content of your files.  I successfully ran my test with the following data:

    "FIELD1","FIELD2","FIELD3","FIELD4","FIELD5",FIELD6,"THEDATETIME",VALUE7,VALUE8
    "ValueA1","ValueA2","ValueA3","ValueA4","ValueA5",100,"Oct-01-2011 0:00:00",100.0, 1000.0
    "ValueB1","ValueB2","ValueB3","ValueB4","ValueB5",101,"Oct-01-2011 0:00:01",101.0, 1001.0

    Also, if you are using MDA 2.1, we now have a MDA Configuration Tool that allows users to generate the necessary metadata by creating an adapter via a UI wizard and applying it to one or more of their batches.  This new tool, which replaces the MDAGenerateMeta.sh script, is documented in the BI 2.1 Information Center under:

    Let us know if you are still having problems.

     

    Regards,
    Lara