Topic
  • 3 replies
  • Latest Post - ‏2012-05-31T08:22:11Z by HienTo
SystemAdmin
SystemAdmin
1245 Posts

Pinned topic Read the whole file using SPL

‏2012-04-30T23:23:16Z |
Hi,

I am a new comer of IBM InfoSphere.
Could anyone tell me how to read the whole file (not each line) as a string then put it into the stream.

Thanks,
Hien
  • mendell
    mendell
    219 Posts

    Re: Read the whole file using SPL

    ‏2012-04-30T23:47:07Z  
    There is a native routine in spl.file::*
    
    
    
    public stateful 
    
    void freadfile (mutable blob res, uint64 file, mutable int32 err)
    

    You can have a Custom operator use fopen, freadfile, fclose to read the whole file. The result is a blob, not an rstring. Is this good enough? It is simple to write a C++ native function that can convert a blob to an rstring.

    Mark
  • askene
    askene
    9 Posts

    Re: Read the whole file using SPL

    ‏2012-05-01T19:17:50Z  
    • mendell
    • ‏2012-04-30T23:47:07Z
    There is a native routine in spl.file::*
    <pre class="jive-pre"> public stateful void freadfile (mutable blob res, uint64 file, mutable int32 err) </pre>
    You can have a Custom operator use fopen, freadfile, fclose to read the whole file. The result is a blob, not an rstring. Is this good enough? It is simple to write a C++ native function that can convert a blob to an rstring.

    Mark
    I've had to do this a few times before. I ended up writing this chunk of code to handle it for me. It was much easier then creating Custom that did the actual file reading.

    The block size I used is the default block size on the machines I use....you may want to change it to whatever your default size is for performance reasons.

    
    
    /** * This returns the contents of an entire file as a single string. */ 
    
    public composite ReadFileAsString ( input file; output stream<rstring result> Out )
    { param expression<rstring> $format: 
    "UTF-8"; graph stream<blob b> filein = FileSource(file) 
    { param format: block; blockSize: 65536u; 
    } stream<rstring result> Out = Custom(filein as IN)  
    { logic state: mutable rstring result = 
    ""; onTuple IN: 
    { result = result + convertToUtf8((list<uint8>)b, $format); 
    } onPunct IN: 
    { 
    
    if (length(result) > 0) 
    { submit(
    {result = result
    }, Out); result = 
    ""; 
    } submit(currentPunct(), Out); 
    } 
    } 
    }
    
  • HienTo
    HienTo
    2 Posts

    Re: Read the whole file using SPL

    ‏2012-05-31T08:22:11Z  
    • askene
    • ‏2012-05-01T19:17:50Z
    I've had to do this a few times before. I ended up writing this chunk of code to handle it for me. It was much easier then creating Custom that did the actual file reading.

    The block size I used is the default block size on the machines I use....you may want to change it to whatever your default size is for performance reasons.

    <pre class="jive-pre"> /** * This returns the contents of an entire file as a single string. */ public composite ReadFileAsString ( input file; output stream<rstring result> Out ) { param expression<rstring> $format: "UTF-8"; graph stream<blob b> filein = FileSource(file) { param format: block; blockSize: 65536u; } stream<rstring result> Out = Custom(filein as IN) { logic state: mutable rstring result = ""; onTuple IN: { result = result + convertToUtf8((list<uint8>)b, $format); } onPunct IN: { if (length(result) > 0) { submit( {result = result }, Out); result = ""; } submit(currentPunct(), Out); } } } </pre>
    Thanks.