Topic
IC4NOTICE: developerWorks Community will be offline May 29-30, 2015 while we upgrade to the latest version of IBM Connections. For more information, read our upgrade FAQ.
13 replies Latest Post - ‏2014-06-18T07:20:29Z by HermannSW
SystemAdmin
SystemAdmin
6772 Posts
ACCEPTED ANSWER

Pinned topic Upper Limit of Maximum Node Size value in XML Parser

‏2010-06-02T08:52:35Z |
Hi all,

Is there any upper limit of Maximum Node Size value in XML Parser(MPGW/XMLFW)?

Thanks in advance.
XI50 (3.8.0.4)
Updated on 2013-03-14T14:48:25Z at 2013-03-14T14:48:25Z by SystemAdmin
  • mandeepk
    mandeepk
    5 Posts
    ACCEPTED ANSWER

    Re: Upper Limit of Maximum Node Size value in XML Parser

    ‏2010-12-08T06:37:25Z  in response to SystemAdmin
    The default size is given in XMLManager-->XMLParser
    XML Maximum Node Size is default(33554432bytes) i.e.,32MB

    regards,
    Sandeep M.
  • SystemAdmin
    SystemAdmin
    6772 Posts
    ACCEPTED ANSWER

    Re: Upper Limit of Maximum Node Size value in XML Parser

    ‏2013-03-12T20:30:36Z  in response to SystemAdmin
    Technically there is no limit in Datapower. In the XML Manager you can set XML Bytes Scanned and XML Maximum Node size to 0 which would allow files to come through of any size. However, keep in mind that the reason the XML Manager is here is to protect against XML threats and to manage what size transactions you send to the backend. For security reasons you will likely want to limit this to whatever your system requires. Additionally, Proxy limits override any similar limit set by the XML manager. You can set "Maximum Message Size" to 0 to allow any size files through. You can also play with your "Proxy parser limits" and "Attachment Byte Count Limit." Hope this helps.
    • HermannSW
      HermannSW
      4493 Posts
      ACCEPTED ANSWER

      Re: Upper Limit of Maximum Node Size value in XML Parser

      ‏2013-03-13T14:38:24Z  in response to SystemAdmin
      > ... Technically there is no limit in Datapower. ...
      >
      There is, but its quite high.
      It is definitely below 78.51GB (total memory, see below) on XI52 models ;-)
      But I once did process a 25GB XML file with a non-streaming stylesheet, so its above 25GB.

      Version: XI52.5.0.0.5 build 223327 on 2013/01/15 14:47:52
      Serial number: 6801610
       
      xi52# show mem
       
          Memory Usage: 3 %
          Total Memory: 82333842 kilobytes
           Used Memory: 2951717 kilobytes
           Free Memory: 79382125 kilobytes
      Requested Memory: 3568000 kilobytes
           Hold Memory: 616283 kilobytes
       Reserved Memory: 16863558 kilobytes
      Installed Memory: 99197400 kilobytes
       
       
      xi52#
      


      Hermann <myXsltBlog/> <myXsltTweets/> <myCE/>
      Updated on 2014-03-25T02:37:58Z at 2014-03-25T02:37:58Z by iron-man
      • SystemAdmin
        SystemAdmin
        6772 Posts
        ACCEPTED ANSWER

        Re: Upper Limit of Maximum Node Size value in XML Parser

        ‏2013-03-13T14:49:12Z  in response to HermannSW
        Hermann,
        That limit is not the same incase of XI50 right? I believe its 4GB

        Regards,
        Kumar
        • HermannSW
          HermannSW
          4493 Posts
          ACCEPTED ANSWER

          Re: Upper Limit of Maximum Node Size value in XML Parser

          ‏2013-03-13T15:01:28Z  in response to SystemAdmin
          Hi Kumar,

          > That limit is not the same incase of XI50 right? I believe its 4GB
          >
          yes, 4GB:
          Version: XI50.4.0.2.9kn build 224171kn on 2013/02/05 21:39:44
          Serial number: 68A3089
           
          xi50# show mem
           
              Memory Usage: 13 %
              Total Memory: 4194304 kilobytes
               Used Memory: 566103 kilobytes
               Free Memory: 3628201 kilobytes
          Requested Memory: 1474688 kilobytes
               Hold Memory: 908585 kilobytes
           
           
          xi50#
          

          Hermann <myXsltBlog/> <myXsltTweets/> <myCE/>
          Updated on 2014-03-25T02:37:53Z at 2014-03-25T02:37:53Z by iron-man
          • SystemAdmin
            SystemAdmin
            6772 Posts
            ACCEPTED ANSWER

            Re: Upper Limit of Maximum Node Size value in XML Parser

            ‏2013-03-13T15:12:28Z  in response to HermannSW
            Hi Hermann,
            So if the appliance is XI50 we can't parse more than 4GB if even XML manager parsing limit is set zero right?
            • HermannSW
              HermannSW
              4493 Posts
              ACCEPTED ANSWER

              Re: Upper Limit of Maximum Node Size value in XML Parser

              ‏2013-03-13T15:19:53Z  in response to SystemAdmin
              Hi,

              > So if the appliance is XI50 we can't parse more than 4GB if even XML manager parsing limit is set zero right?
              >
              that is only true for non-streaming processing.

              I did successfully process 10GB XML files in a streaming service on XI50, there should be no upper limit.

               
              Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>
              • SystemAdmin
                SystemAdmin
                6772 Posts
                ACCEPTED ANSWER

                Re: Upper Limit of Maximum Node Size value in XML Parser

                ‏2013-03-13T15:26:57Z  in response to HermannSW
                Hi Hermann,

                > So if the appliance is XI50 we can't parse more than 4GB if even XML manager parsing limit is set zero right?
                >
                >that is only true for non-streaming processing.

                >I did successfully process 10GB XML files in a streaming service on XI50, there should be no upper limit.

                Can you please explain more about non-steaming and streaming processing?
                Regards,
                Kumar
      • lenisha
        lenisha
        23 Posts
        ACCEPTED ANSWER

        Re: Upper Limit of Maximum Node Size value in XML Parser

        ‏2014-06-17T23:08:19Z  in response to HermannSW

        >> ... Technically there is no limit in Datapower. ...
        >>
        > There is, but its quite high.
        > It is definitely below 78.51GB (total memory, see below) on XI52 models ;-)
        > But I once did process a 25GB XML file with a non-streaming stylesheet, so its above 25GB.
         

        Herman,

        would "xml bytes scanned" setting limit message size even in streaming mode, or it's applicable to non-streaming only?