Topic
  • 3 replies
  • Latest Post - ‏2014-04-23T12:27:09Z by markevans
kennetheld
kennetheld
38 Posts

Pinned topic read file one byte at a time

‏2014-04-22T22:13:29Z |

I am trying to prove that I can read comma delimited files or traditional fixed length files with no record structure hard coded.  What I am attempting to do is read either a comma delimited file or fixed length as a SEQWS type on the associated part and a serial record on the record part.  I would like to read one byte at a time and build my string and then I will determine the field correlations based on soft coded customer specs.  

As I said, I setup the record as a serial record and defined as 1 char thinking I could build an internal string up to the return character and then start over with my next record.  However this is not working.  It seems to be skipping a number of characters for on each record.  

Is my logic TOTALLY flawed?  Can someone give me an idea of what I am doing wrong?  Thanks.

  • markevans
    markevans
    3025 Posts

    Re: read file one byte at a time

    ‏2014-04-22T22:21:04Z  

    Don't think you can do it this way...

    Without seeing your code or trying it, a couple of thoughts:

    a.)  serial files in the distributed world are streams of data... only if CRLF (carriage return/line feed) characters are inserted will there really be separation of lines.   So, I would have thought your char(1) idea would work.  But would need to know what it is "skipping".   You should also not specify TEXT=yes in the resource association. This actually assumes you are reading 3 bytes (1 for data, 2 for CRLF).  It also inserts a CRLF on ADD.

    b.) Why not read a fixed length record defined like this:

    record myrec type SerialRecord

       10 totalrec char(n);

            20 eachbyte char(1)[n];

    end

    then loop through each record after you do the read.  I would think this would be faster since you are doing less physical i/os.

     

    Mark

  • kennetheld
    kennetheld
    38 Posts

    Re: read file one byte at a time

    ‏2014-04-23T00:59:35Z  
    • markevans
    • ‏2014-04-22T22:21:04Z

    Don't think you can do it this way...

    Without seeing your code or trying it, a couple of thoughts:

    a.)  serial files in the distributed world are streams of data... only if CRLF (carriage return/line feed) characters are inserted will there really be separation of lines.   So, I would have thought your char(1) idea would work.  But would need to know what it is "skipping".   You should also not specify TEXT=yes in the resource association. This actually assumes you are reading 3 bytes (1 for data, 2 for CRLF).  It also inserts a CRLF on ADD.

    b.) Why not read a fixed length record defined like this:

    record myrec type SerialRecord

       10 totalrec char(n);

            20 eachbyte char(1)[n];

    end

    then loop through each record after you do the read.  I would think this would be faster since you are doing less physical i/os.

     

    Mark

    Mark,

    I got this working.  The problem I was having was the fact that I had text turned on for the text file I was reading and that was causing characters to be skipped.  I have text turned on for the files I am writing because I want those to terminate with line feeds.

    As for your previous question, the reason I have setup to read one character per read is because we are setting up this interface to be TOTALLY variable on the input side.  I suppose I could come up with something for a record length where I know it would never be exceeded but short of that we would have a hard time defining a length.  

    Anyway, it is currently working.  Thanks for your thoughts and help.  You pointing out the Text yes was key.  

  • markevans
    markevans
    3025 Posts

    Re: read file one byte at a time

    ‏2014-04-23T12:27:09Z  

    Mark,

    I got this working.  The problem I was having was the fact that I had text turned on for the text file I was reading and that was causing characters to be skipped.  I have text turned on for the files I am writing because I want those to terminate with line feeds.

    As for your previous question, the reason I have setup to read one character per read is because we are setting up this interface to be TOTALLY variable on the input side.  I suppose I could come up with something for a record length where I know it would never be exceeded but short of that we would have a hard time defining a length.  

    Anyway, it is currently working.  Thanks for your thoughts and help.  You pointing out the Text yes was key.  

    Glad you got it working.  

     

    f you can define a maximum, you may want to look at using a variable length record (numElementsItem property and/or lengthItem property)....but honestly, not sure I have ever used with Java Gen.   So, your process may be the best and then you don't need to define a maximum length.