Topic
2 replies Latest Post - ‏2006-06-03T12:14:15Z by SystemAdmin
SystemAdmin
SystemAdmin
332 Posts
ACCEPTED ANSWER

Pinned topic Unable to process large XML file

‏2005-02-23T13:12:20Z |
I am using the XML input stage to read in an XML file from a Folder stage and output it to a Sequential file. The job terminates when the file is large (30MB) but it can handle smaller files. I have my folder output set up with the first column as the XML filename and the second column as the XML document itself. I am thinking that the second column cannot handle containing the entire XML document being so large. Is there a way to handle this? I do not know how I can split the file because then the XML Input stage will not be able to parse incomplete bits of XML I do not think. Any suggestions? I was told I could also process these files as a sequential file and do parsing in a transformer. Are there any examples of how that is done?
Thanks.
Updated on 2006-06-03T12:14:15Z at 2006-06-03T12:14:15Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    332 Posts
    ACCEPTED ANSWER

    Re: Unable to process large XML file

    ‏2006-06-02T14:07:49Z  in response to SystemAdmin
    In case it is still relevant

    make your folder stage output only the file name (full path)
    and in the xml use URL instead of column
    HTH
  • SystemAdmin
    SystemAdmin
    332 Posts
    ACCEPTED ANSWER

    Re: Unable to process large XML file

    ‏2006-06-03T12:14:15Z  in response to SystemAdmin
    Hey - hadn't thought of that! Gonna have to give that a shot next time it becomes an issue. :-)