Topic
  • 23 replies
  • Latest Post - ‏2015-06-13T08:19:46Z by Prasad Mhsv
SystemAdmin
SystemAdmin
6772 Posts

Pinned topic Validating JSON?

‏2013-01-02T00:01:34Z |
Is there any easy way out to validate (syntactic & semantic) JSON messages in DataPower? We need to validate rules like mandatory elements, datatypes, field size etc.

RG
Updated on 2013-04-02T13:47:37Z at 2013-04-02T13:47:37Z by swlinn
  • kenhygh
    kenhygh
    1521 Posts

    Re: Validating JSON?

    ‏2013-01-02T01:32:58Z  
    JSON doesn't (yet) have anything like schemas, so if you want to validate it the easiest path is to convert it to XML and validate the XML.

    k
  • SystemAdmin
    SystemAdmin
    6772 Posts

    Re: Validating JSON?

    ‏2013-01-02T08:46:08Z  
    • kenhygh
    • ‏2013-01-02T01:32:58Z
    JSON doesn't (yet) have anything like schemas, so if you want to validate it the easiest path is to convert it to XML and validate the XML.

    k
    Thank you Ken. Will generate XSDs ..

    RG
  • HermannSW
    HermannSW
    4647 Posts

    Re: Validating JSON?

    ‏2013-01-02T14:34:05Z  
    • kenhygh
    • ‏2013-01-02T01:32:58Z
    JSON doesn't (yet) have anything like schemas, so if you want to validate it the easiest path is to convert it to XML and validate the XML.

    k
    Hi Ken,

    > JSON doesn't (yet) have anything like schemas, so if you want to validate it the easiest path is to convert it to XML and validate the XML.
    >
    this is not true, there is a proposed Internet draft "JSON Schema" (JSON Schema is to JSON as XSD is to XML):
    http://json-schema.org/
    Examples page:
    http://json-schema.org/examples.html

    But DataPower firmwares up to 5.0.0.x do not support that JSON Schema validation.

     
    Hermann<myXsltBlog/> <myXsltTweets/>
  • ansmyke
    ansmyke
    79 Posts

    Re: Validating JSON?

    ‏2013-04-02T03:20:54Z  
    • HermannSW
    • ‏2013-01-02T14:34:05Z
    Hi Ken,

    > JSON doesn't (yet) have anything like schemas, so if you want to validate it the easiest path is to convert it to XML and validate the XML.
    >
    this is not true, there is a proposed Internet draft "JSON Schema" (JSON Schema is to JSON as XSD is to XML):
    http://json-schema.org/
    Examples page:
    http://json-schema.org/examples.html

    But DataPower firmwares up to 5.0.0.x do not support that JSON Schema validation.

     
    Hermann<myXsltBlog/> <myXsltTweets/>
    Is there any way in DatapowerXI50 5.0.0.3 to send customized error messagein JSON format for invalid JSON requests?
    The invalid JSON request messages are rejected in the transaction step 0 only before match action is executed and consumers recieves a soap fault instead of JSON fault.
    I understand it would be ridiculous to send invalid JSON requests from the consumer app side in the first hand but I am being curious.

    Thanks,
    Mayank
  • SystemAdmin
    SystemAdmin
    6772 Posts

    Re: Validating JSON?

    ‏2013-04-02T03:47:22Z  
    • ansmyke
    • ‏2013-04-02T03:20:54Z
    Is there any way in DatapowerXI50 5.0.0.3 to send customized error messagein JSON format for invalid JSON requests?
    The invalid JSON request messages are rejected in the transaction step 0 only before match action is executed and consumers recieves a soap fault instead of JSON fault.
    I understand it would be ridiculous to send invalid JSON requests from the consumer app side in the first hand but I am being curious.

    Thanks,
    Mayank
    The way we have done is, we added an ErrorRule, which capture the SOAPFault and then we make a JSONX with our own error code and custom description and before sending back, use store://jsonxtojson.xsl file.

    <json:object xsi:schemaLocation="http://www.datapower.com/schemas/json jsonx.xsd" xmlns:json="http://www.ibm.com/xmlns/prod/2009/jsonx" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <json:object name="Error">
    <json:string name="errorCode">
    <xsl:value-of select="$dpCode"/>
    </json:string>
    <json:string name="errorDesc">
    <xsl:value-of select="Invalid JSON Request"/>
    </json:string>
    </json:object>
    </json:object>

    RG
  • swlinn
    swlinn
    1348 Posts

    Re: Validating JSON?

    ‏2013-04-02T13:47:37Z  
    The way we have done is, we added an ErrorRule, which capture the SOAPFault and then we make a JSONX with our own error code and custom description and before sending back, use store://jsonxtojson.xsl file.

    <json:object xsi:schemaLocation="http://www.datapower.com/schemas/json jsonx.xsd" xmlns:json="http://www.ibm.com/xmlns/prod/2009/jsonx" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <json:object name="Error">
    <json:string name="errorCode">
    <xsl:value-of select="$dpCode"/>
    </json:string>
    <json:string name="errorDesc">
    <xsl:value-of select="Invalid JSON Request"/>
    </json:string>
    </json:object>
    </json:object>

    RG
    I'd assume you could also have a stylesheet with a <xsl:output method="text"> and within the template just output the error message as a string with all of the JSON format, eg, curly braces, etc.

    Regards,
    Steve
  • JTQK_Sunny_Goel
    JTQK_Sunny_Goel
    10 Posts

    Re: Validating JSON?

    ‏2013-10-02T03:57:37Z  
    • HermannSW
    • ‏2013-01-02T14:34:05Z
    Hi Ken,

    > JSON doesn't (yet) have anything like schemas, so if you want to validate it the easiest path is to convert it to XML and validate the XML.
    >
    this is not true, there is a proposed Internet draft "JSON Schema" (JSON Schema is to JSON as XSD is to XML):
    http://json-schema.org/
    Examples page:
    http://json-schema.org/examples.html

    But DataPower firmwares up to 5.0.0.x do not support that JSON Schema validation.

     
    Hermann<myXsltBlog/> <myXsltTweets/>

    Hi Hermann,

    How can we validate Json message in firmware v 6 or higher ? My service has both request and response type as "non-xml". 

    Thanks & Regards

    Sunny Goel

     

  • HermannSW
    HermannSW
    4647 Posts

    Re: Validating JSON?

    ‏2013-10-02T07:55:10Z  

    Hi Hermann,

    How can we validate Json message in firmware v 6 or higher ? My service has both request and response type as "non-xml". 

    Thanks & Regards

    Sunny Goel

     

    Hi Sunny,

    you can just supply JSON Schema Validation (JSV) file (*.jsv) in validate action with 6.0.0.x firmware:
    http://pic.dhe.ibm.com/infocenter/wsdatap/v6r0m0/index.jsp?topic=%2Fcom.ibm.dp.xi.doc%2Fjson_jsonSchemaValidation.html


    From 6.0.0 release notes:
    http://pic.dhe.ibm.com/infocenter/wsdatap/v6r0m0/index.jsp?topic=%2Fcom.ibm.dp.xi.doc%2FrelnotesXI.html

    JSON schema validation
    Supports JSON schema validation, draft specification version 03, A JSON Media Type for Describing the Structure and Meaning of JSON Documents. The specification is also available as a self-describing JSON schema.



    There are quite some online JSON Schema validators you may use to play with JSV:

    https://www.google.com/search?q=json+schema+validation+online


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

    Updated on 2013-10-02T08:01:01Z at 2013-10-02T08:01:01Z by HermannSW
  • Rohit-Goyal
    Rohit-Goyal
    133 Posts

    Re: Validating JSON?

    ‏2014-05-08T00:31:31Z  
    • HermannSW
    • ‏2013-10-02T07:55:10Z

    Hi Sunny,

    you can just supply JSON Schema Validation (JSV) file (*.jsv) in validate action with 6.0.0.x firmware:
    http://pic.dhe.ibm.com/infocenter/wsdatap/v6r0m0/index.jsp?topic=%2Fcom.ibm.dp.xi.doc%2Fjson_jsonSchemaValidation.html


    From 6.0.0 release notes:
    http://pic.dhe.ibm.com/infocenter/wsdatap/v6r0m0/index.jsp?topic=%2Fcom.ibm.dp.xi.doc%2FrelnotesXI.html

    JSON schema validation
    Supports JSON schema validation, draft specification version 03, A JSON Media Type for Describing the Structure and Meaning of JSON Documents. The specification is also available as a self-describing JSON schema.



    There are quite some online JSON Schema validators you may use to play with JSV:

    https://www.google.com/search?q=json+schema+validation+online


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

    Hi Herman,

    I am trying to implement JSON schema validation. For some reason I am not able to successfully implement it. I tired the JSON schema available here.. 

    http://pic.dhe.ibm.com/infocenter/wsdatap/v6r0m0/index.jsp?topic=%2Fcom.ibm.dp.xm.doc%2Fjson_jsonexamples.html

    And tried a valid JSON 

    DataPower keep throwing error.. 

     
    10:15:50 AM mpgw error 70577287 error 10.186.115.85 0x00c30010 mpgw (Facade01): Internal Error
    10:15:50 AM multistep error 70577287 request 10.186.115.85 0x80c00009
    mpgw (Facade01): request Facade01_Policy_rule_0 #1 validate: 'INPUT json-schema local:///ondisk/POC/JsonSchema.jsv' failed: Internal Error

    To confirm if my JSON schema and JSON message are valid, I tried on http://jsonschemalint.com/ and it's says all OK

    In validation action, I have used "Validate Document via JSON Schema URL" as Schema Validate Method.

    Can you suggest where possible I am doing wrong?

    Regards,

    Rohit Goyal

    Updated on 2014-05-08T00:32:10Z at 2014-05-08T00:32:10Z by Rohit-Goyal
  • HermannSW
    HermannSW
    4647 Posts

    Re: Validating JSON?

    ‏2014-05-08T12:53:31Z  

    Hi Herman,

    I am trying to implement JSON schema validation. For some reason I am not able to successfully implement it. I tired the JSON schema available here.. 

    http://pic.dhe.ibm.com/infocenter/wsdatap/v6r0m0/index.jsp?topic=%2Fcom.ibm.dp.xm.doc%2Fjson_jsonexamples.html

    And tried a valid JSON 

    DataPower keep throwing error.. 

     
    10:15:50 AM mpgw error 70577287 error 10.186.115.85 0x00c30010 mpgw (Facade01): Internal Error
    10:15:50 AM multistep error 70577287 request 10.186.115.85 0x80c00009
    mpgw (Facade01): request Facade01_Policy_rule_0 #1 validate: 'INPUT json-schema local:///ondisk/POC/JsonSchema.jsv' failed: Internal Error

    To confirm if my JSON schema and JSON message are valid, I tried on http://jsonschemalint.com/ and it's says all OK

    In validation action, I have used "Validate Document via JSON Schema URL" as Schema Validate Method.

    Can you suggest where possible I am doing wrong?

    Regards,

    Rohit Goyal

    Hi Rohit,

    what is your sevrice request type?
    JSV does only work if request type is JSON.


    Hermann.

  • Rohit-Goyal
    Rohit-Goyal
    133 Posts

    Re: Validating JSON?

    ‏2014-05-08T13:02:58Z  
    • HermannSW
    • ‏2014-05-08T12:53:31Z

    Hi Rohit,

    what is your sevrice request type?
    JSV does only work if request type is JSON.


    Hermann.

    Thanks Herman. I was trying with "non-xml". I will try with JSON request type and will confirm. Is there any specific reason why it require request type as JSON?

    Does that also mean that XSD validation won't work if Request Type is non-xml? I never tried this, just confirming. 

    - Rohit

  • HermannSW
    HermannSW
    4647 Posts

    Re: Validating JSON?

    ‏2014-05-08T14:48:56Z  

    Thanks Herman. I was trying with "non-xml". I will try with JSON request type and will confirm. Is there any specific reason why it require request type as JSON?

    Does that also mean that XSD validation won't work if Request Type is non-xml? I never tried this, just confirming. 

    - Rohit

    > Does that also mean that XSD validation won't work if Request Type is non-xml? I never tried this, just confirming.
    >
    I never did that, but that cannot work.

    With Non-XML request type the input (XML) does not get parsed by XML Parser, and so the XSD validation has no input to validate ...


    Hermann.

  • RiyazShah
    RiyazShah
    56 Posts

    Re: Validating JSON?

    ‏2014-09-08T15:57:44Z  

    Thanks Herman. I was trying with "non-xml". I will try with JSON request type and will confirm. Is there any specific reason why it require request type as JSON?

    Does that also mean that XSD validation won't work if Request Type is non-xml? I never tried this, just confirming. 

    - Rohit

    Hi,

    I have a same scenario to validate the incoming JSON request. I am trying the same thing but my json is schema is not validating the request. I might be wrong with the schema. Could you please provide any sample request and json schema to validate.

     

    Thanks!

    Riyaz

  • kenhygh
    kenhygh
    1521 Posts

    Re: Validating JSON?

    ‏2014-09-08T22:55:45Z  
    • RiyazShah
    • ‏2014-09-08T15:57:44Z

    Hi,

    I have a same scenario to validate the incoming JSON request. I am trying the same thing but my json is schema is not validating the request. I might be wrong with the schema. Could you please provide any sample request and json schema to validate.

     

    Thanks!

    Riyaz

    This is trivial:

    { "bankNumber":2, "accountNumber": 4, "productCode": "b" }

    "description": "Taken from Account.xsd",
    "type": "object",
    "properties": {
    "accountNumber": { "type": "integer", "required": true },
    "productCode": { "type": "string", "required": true },
    "bankNumber": { "type": "integer", "required": true }
    }
    }
  • RiyazShah
    RiyazShah
    56 Posts

    Re: Validating JSON?

    ‏2014-09-09T04:03:50Z  
    • kenhygh
    • ‏2014-09-08T22:55:45Z

    This is trivial:

    { "bankNumber":2, "accountNumber": 4, "productCode": "b" }

    "description": "Taken from Account.xsd",
    "type": "object",
    "properties": {
    "accountNumber": { "type": "integer", "required": true },
    "productCode": { "type": "string", "required": true },
    "bankNumber": { "type": "integer", "required": true }
    }
    }

    Hi kenhygh,

    thank you for your reply. 

    I have my JSON request is looks like, 

    {
        "Employee": {
            "ID": "123323654",
            "Number": "222",
            "Location": "8",
            "Data": [
                {
                    "Course_Code": "AT",
                    "GPA": "3.3",
                    "UniqueId": "123654",
                    "version": "5.0"
                }
            ]
        }
    }

     

    In those Data---- "Course_Code" is like optional.. Could you please help me how to create json schema for this. 

    Thanks for your time.

  • kenhygh
    kenhygh
    1521 Posts

    Re: Validating JSON?

    ‏2014-09-09T12:30:47Z  
    • RiyazShah
    • ‏2014-09-09T04:03:50Z

    Hi kenhygh,

    thank you for your reply. 

    I have my JSON request is looks like, 

    {
        "Employee": {
            "ID": "123323654",
            "Number": "222",
            "Location": "8",
            "Data": [
                {
                    "Course_Code": "AT",
                    "GPA": "3.3",
                    "UniqueId": "123654",
                    "version": "5.0"
                }
            ]
        }
    }

     

    In those Data---- "Course_Code" is like optional.. Could you please help me how to create json schema for this. 

    Thanks for your time.

    After some googling around, I came up with this:

    {
    "type": "object",
    "required": [ "Employee" ],
    "properties": {
        "Employee": {
          "type": "object",
          "required": ["ID", "Number", "Location", "Data" ],
          "properties": {
             "ID": { 
                "type": "string",
                "enum": ["123323654"]
              },
              "Number": {
                 "type": "string",
                 "enum": ["222"]
              },
              "Location": {
                 "type": "string",
                 "enum": ["8"]
              },
              "Data": {
                 "type": "array",
                 "items": { 
                    "type": "object",
                    "required": ["GPA","UniqueId","version"],
                    "properties": {
                       "Course_Code": {
                          "type": "string"
                        },
                       "GPA": { "type": "string", "enum": ["3.3"] },
                       "UniqueId": { "type": "string", "enum": ["123654"] },
                       "version": {"type":"string", "enum": ["5.0"]}
                    }
                  },
                 "minItems": 1
                }
           }
        }
      }
    }
  • swlinn
    swlinn
    1348 Posts

    Re: Validating JSON?

    ‏2014-09-09T17:42:30Z  
    • RiyazShah
    • ‏2014-09-09T04:03:50Z

    Hi kenhygh,

    thank you for your reply. 

    I have my JSON request is looks like, 

    {
        "Employee": {
            "ID": "123323654",
            "Number": "222",
            "Location": "8",
            "Data": [
                {
                    "Course_Code": "AT",
                    "GPA": "3.3",
                    "UniqueId": "123654",
                    "version": "5.0"
                }
            ]
        }
    }

     

    In those Data---- "Course_Code" is like optional.. Could you please help me how to create json schema for this. 

    Thanks for your time.

    I've used this site in the past:  http://jsonschema.net/ ... You can provide a schema and have it generate a sample JSON document or visa versa.  Beware of your firmware version and the schema produced as DataPower 6.0 supported on version of the schema and 7.0 added support for a later version of the schema, so you need to ensure your produced schema is supported by your appliance firmware.

    Regards,

    Steve

  • ipt_clu
    ipt_clu
    14 Posts

    Re: Validating JSON?

    ‏2014-09-10T13:58:28Z  
    • swlinn
    • ‏2014-09-09T17:42:30Z

    I've used this site in the past:  http://jsonschema.net/ ... You can provide a schema and have it generate a sample JSON document or visa versa.  Beware of your firmware version and the schema produced as DataPower 6.0 supported on version of the schema and 7.0 added support for a later version of the schema, so you need to ensure your produced schema is supported by your appliance firmware.

    Regards,

    Steve

    Please be aware that json schema draft-03 syntax for "required" is different from draft-04:

    draft-03:

    The "required" property is a child of the actual property

     

        "Employee": {
          "type": "object",      
          "properties": {
             "ID":
                "type": "string",
                "required": true,

     

    whereas in draft-04, the "required" property is a property (of type array) of the parent:

    "Employee": {
          "type": "object",
          "required": ["ID", "Number", "Location", "Data" ],

     

    DataPower 6 supports only draft 03 and DataPower 7.0.0 uses schema draft-03 by default. If you want to use the schema draft-0.4 in DataPower 7.0.0+, you need to explicitely say so in your schema:

    {

      "$schema":"http://json-schema.org/draft-04/schema#",
      "title": "My Schema",
      "type": "object",

      ...

    Additionally, please be aware that additional properties are allowed by default. If you want / need to restrict the properties, add "additionalProperties": false to your object.

    Updated on 2014-09-10T14:02:52Z at 2014-09-10T14:02:52Z by ipt_clu
  • ipt_clu
    ipt_clu
    14 Posts

    Re: Validating JSON?

    ‏2014-09-10T14:02:24Z  
    • kenhygh
    • ‏2014-09-09T12:30:47Z

    After some googling around, I came up with this:

    {
    "type": "object",
    "required": [ "Employee" ],
    "properties": {
        "Employee": {
          "type": "object",
          "required": ["ID", "Number", "Location", "Data" ],
          "properties": {
             "ID": { 
                "type": "string",
                "enum": ["123323654"]
              },
              "Number": {
                 "type": "string",
                 "enum": ["222"]
              },
              "Location": {
                 "type": "string",
                 "enum": ["8"]
              },
              "Data": {
                 "type": "array",
                 "items": { 
                    "type": "object",
                    "required": ["GPA","UniqueId","version"],
                    "properties": {
                       "Course_Code": {
                          "type": "string"
                        },
                       "GPA": { "type": "string", "enum": ["3.3"] },
                       "UniqueId": { "type": "string", "enum": ["123654"] },
                       "version": {"type":"string", "enum": ["5.0"]}
                    }
                  },
                 "minItems": 1
                }
           }
        }
      }
    }

    Off topic: I am not sure it's a good idea to use enum for employee ID, unless you want to maintain all employee IDs in your schema. The same may hold true for number (depends on what number means). But I would definitively use it in location.

  • kenhygh
    kenhygh
    1521 Posts

    Re: Validating JSON?

    ‏2014-09-10T14:59:09Z  
    • ipt_clu
    • ‏2014-09-10T14:02:24Z

    Off topic: I am not sure it's a good idea to use enum for employee ID, unless you want to maintain all employee IDs in your schema. The same may hold true for number (depends on what number means). But I would definitively use it in location.

    I'm sure it's a horrible idea. But it's what the requirement seems to be.

  • Prasad Mhsv
    Prasad Mhsv
    22 Posts

    Re: Validating JSON?

    ‏2015-06-12T12:30:05Z  

    Hi,

     

    I need to convert my json message to json schema draft 3 version. could you please suggest some tool where i can get this done . 

    Tried looking for some online convertor/tools but all are generating the schema in draft 4 version. 

    I am working on datapower version 6 which supports only draft 03 version.

     

    Thanks,

    Prasad

  • HermannSW
    HermannSW
    4647 Posts

    Re: Validating JSON?

    ‏2015-06-12T14:03:41Z  

    Hi,

     

    I need to convert my json message to json schema draft 3 version. could you please suggest some tool where i can get this done . 

    Tried looking for some online convertor/tools but all are generating the schema in draft 4 version. 

    I am working on datapower version 6 which supports only draft 03 version.

     

    Thanks,

    Prasad

    I have not done that, but what about this procedure:

    1. generate JSV by a tool you found
    2. use that JSV (draft-4) on v6 and check for errors reported
    3. if no error, you are done
    4. if error, manipulate the JSV to remove the reported issue
    5. goto 2

     

    Hermann

    Updated on 2015-06-12T14:08:17Z at 2015-06-12T14:08:17Z by HermannSW
  • Prasad Mhsv
    Prasad Mhsv
    22 Posts

    Re: Validating JSON?

    ‏2015-06-13T08:19:46Z  
    • HermannSW
    • ‏2015-06-12T14:03:41Z

    I have not done that, but what about this procedure:

    1. generate JSV by a tool you found
    2. use that JSV (draft-4) on v6 and check for errors reported
    3. if no error, you are done
    4. if error, manipulate the JSV to remove the reported issue
    5. goto 2

     

    Hermann

    Thank you Hermann,