Topic
  • 2 replies
  • Latest Post - ‏2006-06-03T12:14:15Z by SystemAdmin
SystemAdmin
SystemAdmin
332 Posts

Pinned topic Unable to process large XML file

‏2005-02-23T13:12:20Z |
I am using the XML input stage to read in an XML file from a Folder stage and output it to a Sequential file. The job terminates when the file is large (30MB) but it can handle smaller files. I have my folder output set up with the first column as the XML filename and the second column as the XML document itself. I am thinking that the second column cannot handle containing the entire XML document being so large. Is there a way to handle this? I do not know how I can split the file because then the XML Input stage will not be able to parse incomplete bits of XML I do not think. Any suggestions? I was told I could also process these files as a sequential file and do parsing in a transformer. Are there any examples of how that is done?
Thanks.
Updated on 2006-06-03T12:14:15Z at 2006-06-03T12:14:15Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    332 Posts

    Re: Unable to process large XML file

    ‏2006-06-02T14:07:49Z  
    In case it is still relevant

    make your folder stage output only the file name (full path)
    and in the xml use URL instead of column
    HTH
  • SystemAdmin
    SystemAdmin
    332 Posts

    Re: Unable to process large XML file

    ‏2006-06-03T12:14:15Z  
    Hey - hadn't thought of that! Gonna have to give that a shot next time it becomes an issue. :-)