Topic
  • 2 replies
  • Latest Post - ‏2015-11-04T15:35:03Z by Stefan_Heublein
Stefan_Heublein
Stefan_Heublein
7 Posts

Pinned topic Named Pipes vs. IBM COBOL for AIX V4.1.1

‏2014-03-11T15:44:41Z |

Hi all,

Does someone know how to write to (and read from) a named pipe using IBM COBOL for AIX V4.1.1?

Basically we want to write a line sequential file into a named pipe. But OPEN always fails with file status code 39. The same happens if we try to read from a named pipe.

Please find below some COBOL code writing a line sequential file:

000100 ID DIVISION.                                                     00010002
000200 PROGRAM-ID. PIPEPGM.                                             00020002
000300                                                                  00030002
000400 ENVIRONMENT DIVISION.                                            00040002
000500 INPUT-OUTPUT SECTION.                                            00050002
000600                                                                  00060002
000700 FILE-CONTROL.                                                    00070002
000800                                                                  00080002
000900     SELECT PIPEFILE ASSIGN TO 'mypipe'                           00090002
001000     FILE STATUS IS W-STAT-PIPEFILE                               00100002
001010     ORGANIZATION IS LINE SEQUENTIAL.                            
001100                                                                  00110002
001200 DATA DIVISION.                                                   00120002
001300 FILE SECTION.                                                    00130002
001400                                                                  00140002
001500 FD   PIPEFILE                                                    00150002
001600      LABEL RECORDS ARE STANDARD                                  00160002
001700      RECORDING MODE IS F                                         00170002
001800      BLOCK CONTAINS 0 RECORDS                                    00180002
001900      DATA  RECORD   IS PIPEFILE-REC.                             00190002
002000                                                                  00200002
002100 01   PIPEFILE-REC                  PIC X(080).                   00210002
002200                                                                  00220002
002300 WORKING-STORAGE SECTION.                                         00230002
002400                                                                  00240002
002500 01  W-STAT-FELDER.                                               00250002
002600     05 W-STAT-PIPEFILE             PIC  X(02).                   00260002
002700        88 W-STAT-PIPEFILE-OK       VALUE  '00'.                  00270002
002800                                                                  00280002
002900 01  W-L-WRITE-PIPEFILE             PIC  X(80) VALUE              00290002
003000     'WRITE PIPEFILE TEST'.                                       00300002
003100                                                                  00310002
003200 PROCEDURE DIVISION.                                              00320002
003300                                                                  00330002
003400 S0000-STEUERUNG SECTION.                                         00340002
003500                                                                  00350002
003600     PERFORM F0100-OPEN-PIPEFILE                                  00360002
003700     PERFORM M0000-MAIN                                           00370002
003800     PERFORM F0300-CLOSE-PIPEFILE                                 00380002
003900                                                                  00390002
004000     GOBACK.                                                      00400002
004100                                                                  00410002
004200 M0000-MAIN SECTION.                                              00420002
004300                                                                  00430002
004400     PERFORM  F0200-WRITE-PIPEFILE                                00440002
004600                                                                  00460002
004700     EXIT.                                                        00470002
004800                                                                  00480002
004900 F0100-OPEN-PIPEFILE SECTION.                                     00490002
005000                                                                  00500002
005100     OPEN OUTPUT PIPEFILE                                         00510002
005200                                                                  00520002
005300     EVALUATE TRUE                                                00530002
005400        WHEN W-STAT-PIPEFILE-OK                                   00540002
005500           CONTINUE                                               00550002
005600        WHEN OTHER                                                00560002
005700           DISPLAY 'INVALID PIPEFILE OPEN ST: ' W-STAT-PIPEFILE   00570002
005800     END-EVALUATE                                                 00580002
005900                                                                  00590002
006000     EXIT.                                                        00600002
006100                                                                  00610002
006200 F0200-WRITE-PIPEFILE SECTION.                                    00620002
006300                                                                  00630002
006400     WRITE PIPEFILE-REC FROM W-L-WRITE-PIPEFILE                   00640002
006500                                                                  00650002
006600     EVALUATE TRUE                                                00660002
006700        WHEN W-STAT-PIPEFILE-OK                                   00670002
006800           CONTINUE                                               00680002
006900        WHEN OTHER                                                00690002
007000           DISPLAY 'INVALID PIPEFILE WRITE ST: ' W-STAT-PIPEFILE  00700002
007100     END-EVALUATE                                                 00710002
007200                                                                  00720002
007300     EXIT.                                                        00730002
007400                                                                  00740002
007500 F0300-CLOSE-PIPEFILE SECTION.                                    00750002
007600                                                                  00760002
007700     CLOSE  PIPEFILE                                              00770002
007800                                                                  00780002
007900     EVALUATE TRUE                                                00790002
008000        WHEN W-STAT-PIPEFILE-OK                                   00800002
008100           CONTINUE                                               00810002
008200        WHEN OTHER                                                00820002
008300           DISPLAY 'INVALID PIPEFILE CLOSE ST: ' W-STAT-PIPEFILE  00830002
008400     END-EVALUATE                                                 00840002
008500                                                                  00850002
008600                                                                  00860002
008700     EXIT.                                                        00870002
008800                                                                  00880002
008900 END PROGRAM PIPEPGM.                                             00890002

 

Test procedure on AIX 7.7:

> mkfifo mypipe

> cat mypipe &

> ./PIPEPGM

Output:

INVALID PIPEFILE OPEN ST: 39

What are we doing wrong?

Any ideas?

BTW: Switching from LSQ to STL files does not solve the issue.

Thank you in advance.

Best regards from Germany

Stefan

Updated on 2014-03-11T16:26:52Z at 2014-03-11T16:26:52Z by Stefan_Heublein
  • outlaw
    outlaw
    41 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2014-03-25T20:39:32Z  

    named pipes (fifo) have a few oddities that make them a we bit o' pain...

    A file system with metadata (STL, SdU, SFS, DB2) will never work over a pipe - due to the, potentially out of band, metadata.
    I would not want to support Line Sequential (LSQ) over a pipe - do to the non-record based nature of LSQ (ie: short reads).

    That would seem to leave us with RSD and QSAM (used to be RAW).    Both sides of the pipe would have to agree on both the file system, and the record format (fixed vs variable lengths) - or communication would utterly futile !  Note that without metadata, we can't really verify matching conditions!

    Now, we get to the fun stuff !
    1) Who logically owns the pipe (reader, writer, first come)?
    2) Do we convert a 'WRITE ONLY' open (OUTPUT) to a 'READ/WRITE' (I/O) if the former would block?
    3) Do we stick with non-blocking I/O, and introduce/reuse a file status field to say 'request would block, try again' ?
    4) If we move to blocking I/O, how long do we wait, and do we limit the retries ?

    TL;DR -

    0)  What is it you are trying to accomplish here, and what (if any) functionality do you expect from a FIFO vs a regular file ?
    1)  Do you need named pipe (mkfifo(1)), or would a regular pipe (pipe(2)) be sufficient ?
    2)  Is this always a one-to-one (reader-writer) setup, or will you need multiple sources and/or sinks ?
    3)  Is this new functionality, or did a prior COBOL environment support this ?
    4) What is  your best shot a trying to sell the need for this feature (to those who prioritize things) ?
    5) Would you be willing to test, validate, and comment on a implementation ?

     

     

     

  • Stefan_Heublein
    Stefan_Heublein
    7 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2014-03-26T18:58:28Z  
    • outlaw
    • ‏2014-03-25T20:39:32Z

    named pipes (fifo) have a few oddities that make them a we bit o' pain...

    A file system with metadata (STL, SdU, SFS, DB2) will never work over a pipe - due to the, potentially out of band, metadata.
    I would not want to support Line Sequential (LSQ) over a pipe - do to the non-record based nature of LSQ (ie: short reads).

    That would seem to leave us with RSD and QSAM (used to be RAW).    Both sides of the pipe would have to agree on both the file system, and the record format (fixed vs variable lengths) - or communication would utterly futile !  Note that without metadata, we can't really verify matching conditions!

    Now, we get to the fun stuff !
    1) Who logically owns the pipe (reader, writer, first come)?
    2) Do we convert a 'WRITE ONLY' open (OUTPUT) to a 'READ/WRITE' (I/O) if the former would block?
    3) Do we stick with non-blocking I/O, and introduce/reuse a file status field to say 'request would block, try again' ?
    4) If we move to blocking I/O, how long do we wait, and do we limit the retries ?

    TL;DR -

    0)  What is it you are trying to accomplish here, and what (if any) functionality do you expect from a FIFO vs a regular file ?
    1)  Do you need named pipe (mkfifo(1)), or would a regular pipe (pipe(2)) be sufficient ?
    2)  Is this always a one-to-one (reader-writer) setup, or will you need multiple sources and/or sinks ?
    3)  Is this new functionality, or did a prior COBOL environment support this ?
    4) What is  your best shot a trying to sell the need for this feature (to those who prioritize things) ?
    5) Would you be willing to test, validate, and comment on a implementation ?

     

     

     

    Thank you for your response and comments.

    Please find below answers to some of your questions.

    1) Who logically owns the pipe (reader, writer, first come)?

    I would say the one who first opens the pipe.


    0) What is it you are trying to accomplish here, and what (if any) functionality do you expect from a FIFO vs a regular file ?

    Piping the contents of COBOL output files (RSD, LSQ or QSAM) directly into other applications or tools like Dovetailed Technologies Co:Z Dataset Pipes (please refer to https://dovetail.com/products/dspipes.html ).

    Of course the consumer / reader of the pipe data can be non-COBOL, too.

    BTW:

    DISPLAY UPON SYSPUNCH (please refer to https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014509411 ) can already be used to write to a named pipe. Just export the environment variable SYSPUNCH set to the name of the named pipe. This works without any problem. From my point of view this is a good example how it should work with any RSD, LSQ or QSAM output file, too.


    1) Do you need named pipe (mkfifo(1)), or would a regular pipe (pipe(2)) be sufficient ?

    Both would be O.K. in our scenario. We can work with both of them.


    2) Is this always a one-to-one (reader-writer) setup, or will you need multiple sources and/or sinks ?

    Our use case it is a one2one setup, only.


    3) Is this new functionality, or did a prior COBOL environment support this ?

    Reading the IBM COBOL for AIX related article https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014614797 I would guess that this worked with an earlier IBM COBOL for AIX version… ;-)


    4) What is your best shot a trying to sell the need for this feature (to those who prioritize things) ?

    Other COBOL compilers like Micro Focus COBOL and OpenCOBOL support this out the box…

    For an OpenCOBOL related example please refer to:

    http://opencobol.add1tocobol.com/#can-opencobol-handle-named-pipes


    5) Would you be willing to test, validate, and comment on a implementation ?

    Yes, of course.

    Thank you again.

    Best regards from Germany

    Stefan

  • outlaw
    outlaw
    41 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2014-03-26T21:09:06Z  

    Thank you for your response and comments.

    Please find below answers to some of your questions.

    1) Who logically owns the pipe (reader, writer, first come)?

    I would say the one who first opens the pipe.


    0) What is it you are trying to accomplish here, and what (if any) functionality do you expect from a FIFO vs a regular file ?

    Piping the contents of COBOL output files (RSD, LSQ or QSAM) directly into other applications or tools like Dovetailed Technologies Co:Z Dataset Pipes (please refer to https://dovetail.com/products/dspipes.html ).

    Of course the consumer / reader of the pipe data can be non-COBOL, too.

    BTW:

    DISPLAY UPON SYSPUNCH (please refer to https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014509411 ) can already be used to write to a named pipe. Just export the environment variable SYSPUNCH set to the name of the named pipe. This works without any problem. From my point of view this is a good example how it should work with any RSD, LSQ or QSAM output file, too.


    1) Do you need named pipe (mkfifo(1)), or would a regular pipe (pipe(2)) be sufficient ?

    Both would be O.K. in our scenario. We can work with both of them.


    2) Is this always a one-to-one (reader-writer) setup, or will you need multiple sources and/or sinks ?

    Our use case it is a one2one setup, only.


    3) Is this new functionality, or did a prior COBOL environment support this ?

    Reading the IBM COBOL for AIX related article https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014614797 I would guess that this worked with an earlier IBM COBOL for AIX version… ;-)


    4) What is your best shot a trying to sell the need for this feature (to those who prioritize things) ?

    Other COBOL compilers like Micro Focus COBOL and OpenCOBOL support this out the box…

    For an OpenCOBOL related example please refer to:

    http://opencobol.add1tocobol.com/#can-opencobol-handle-named-pipes


    5) Would you be willing to test, validate, and comment on a implementation ?

    Yes, of course.

    Thank you again.

    Best regards from Germany

    Stefan

    Thanks for the feedback and additional information !

    The native file systems were completely rewritten in the 4.1.1.x series to provide better performance, throughput, and RAS.
    It is quite possible that something slipped in that doesn't play well with fifos - even though an effort was made to
    fully support fifos (hopefully with enhancements coming later).

    The fact that DISPLAY upon SYSPUNCH works, well - likely any DISPLAY unit, is that section of code has not, yet, been updated - and typically does a fflush() after every DISPLAY, which also helps with a pipe, but can cause other performance issues.

    I will check to see what is going on!

     

  • outlaw
    outlaw
    41 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2014-03-31T19:33:32Z  
    • outlaw
    • ‏2014-03-26T21:09:06Z

    Thanks for the feedback and additional information !

    The native file systems were completely rewritten in the 4.1.1.x series to provide better performance, throughput, and RAS.
    It is quite possible that something slipped in that doesn't play well with fifos - even though an effort was made to
    fully support fifos (hopefully with enhancements coming later).

    The fact that DISPLAY upon SYSPUNCH works, well - likely any DISPLAY unit, is that section of code has not, yet, been updated - and typically does a fflush() after every DISPLAY, which also helps with a pipe, but can cause other performance issues.

    I will check to see what is going on!

     

    Can you open a PMR on this issue 'OPEN fails on a named pipe' ?

    There is a good chance we can get this fixed in the next PTF

  • texjim
    texjim
    3 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2014-06-09T01:24:47Z  

    I've recently encountered this same problem when migrating Cobol for AIX version 4.1.0.3 to 4.1.1.10. 

    I've been successfully using pipes to pass 'records' via a Line sequential pipe file for 3 to 4 years with version 4.1.0.3

    Our usage is a one to one relationship  - 1 writer to 1 reader. 

    My question today is - was a PMR opened for this and what would be the anticipated time frame for the next PTF.

    If you need more information on my usage or have any other questions I can provide them.

     

  • Stefan_Heublein
    Stefan_Heublein
    7 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2014-06-10T08:39:57Z  
    • texjim
    • ‏2014-06-09T01:24:47Z

    I've recently encountered this same problem when migrating Cobol for AIX version 4.1.0.3 to 4.1.1.10. 

    I've been successfully using pipes to pass 'records' via a Line sequential pipe file for 3 to 4 years with version 4.1.0.3

    Our usage is a one to one relationship  - 1 writer to 1 reader. 

    My question today is - was a PMR opened for this and what would be the anticipated time frame for the next PTF.

    If you need more information on my usage or have any other questions I can provide them.

     

    Hi,

    I haven't opened a PMR for this problem so far.

    We have officially licensed the IBM COBOL for AIX product but due to internal reasons we're not able to open PMRs.

    So, describing the problem in this public forum is the only thing I can do.

     

    BTW:

    We found some problems in the integrated DB2 coprocessor, too. E.g. it is able to compile the SET CURRENT PACKAGE PATH statement of DB2, but at runtime it doesn't work (using DB2 Connect and DB2 for z/OS V8 NFM, works fine with external DB2 precompiler). And all Underscore characters  ("_") in SQL options for the DB2 coprocessor are replaced by a different character which prevents programmers from using PRECOMPILE options like DEFERRED_PREPARE.  

     

    Best regards from Germany

    Stefan

     

     

     

    Updated on 2014-06-10T09:16:29Z at 2014-06-10T09:16:29Z by Stefan_Heublein
  • outlaw
    outlaw
    41 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2014-06-10T15:34:44Z  
    • texjim
    • ‏2014-06-09T01:24:47Z

    I've recently encountered this same problem when migrating Cobol for AIX version 4.1.0.3 to 4.1.1.10. 

    I've been successfully using pipes to pass 'records' via a Line sequential pipe file for 3 to 4 years with version 4.1.0.3

    Our usage is a one to one relationship  - 1 writer to 1 reader. 

    My question today is - was a PMR opened for this and what would be the anticipated time frame for the next PTF.

    If you need more information on my usage or have any other questions I can provide them.

     

    Unfortunately, no - no PMR has been opened.   That means that the fix will not make 4.1.1.11

    The support is all there for simple usage, but a test at open time (geared for future function) mistakenly prevents any PIPE/FIFO

  • texjim
    texjim
    3 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2014-09-03T20:23:03Z  
    • outlaw
    • ‏2014-06-10T15:34:44Z

    Unfortunately, no - no PMR has been opened.   That means that the fix will not make 4.1.1.11

    The support is all there for simple usage, but a test at open time (geared for future function) mistakenly prevents any PIPE/FIFO

    Opened a PMR 93569,7TD,000 for this in June.

    Have been getting updates every couple weeks of  "Working on it.  Need more time."

  • Stefan_Heublein
    Stefan_Heublein
    7 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2014-09-04T13:56:26Z  
    • texjim
    • ‏2014-09-03T20:23:03Z

    Opened a PMR 93569,7TD,000 for this in June.

    Have been getting updates every couple weeks of  "Working on it.  Need more time."

    Hi,

    We have opened PMR "52674,075,724 - OPEN fails on a named pipe" on 23th of June 2014.

    I'm currently wainting for an answer from IBM if there's a fix targeted for the end of October 2014 PTF for COBOL for AIX, V4.1.1.

    Best regards from Germany

    Stefan

     

  • outlaw
    outlaw
    41 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2014-09-09T16:30:42Z  
    • outlaw
    • ‏2014-06-10T15:34:44Z

    Unfortunately, no - no PMR has been opened.   That means that the fix will not make 4.1.1.11

    The support is all there for simple usage, but a test at open time (geared for future function) mistakenly prevents any PIPE/FIFO

    Are you both only writing to a pipe/fifo - or do you also read from one (in COBOL) ?

  • Stefan_Heublein
    Stefan_Heublein
    7 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2014-09-10T14:55:34Z  
    • outlaw
    • ‏2014-09-09T16:30:42Z

    Are you both only writing to a pipe/fifo - or do you also read from one (in COBOL) ?

    Hi,

    At the moment we write to a named pipe, only.

    Best regards from Germany

    Stefan

  • Stefan_Heublein
    Stefan_Heublein
    7 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2014-09-16T15:46:28Z  

    Hi,

    We have opened PMR "52674,075,724 - OPEN fails on a named pipe" on 23th of June 2014.

    I'm currently wainting for an answer from IBM if there's a fix targeted for the end of October 2014 PTF for COBOL for AIX, V4.1.1.

    Best regards from Germany

    Stefan

     

    Hi,
    I have received a PMR status update from IBM today.
    They are currently working on a fix but this requires more time.
    So, the fix will not be included in the October 2014 PTF.
    But an interim fix will be made available once the fix is complete.
    Best regards from Germany
    Stefan
  • texjim
    texjim
    3 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2015-11-03T20:44:29Z  
    Hi,
    I have received a PMR status update from IBM today.
    They are currently working on a fix but this requires more time.
    So, the fix will not be included in the October 2014 PTF.
    But an interim fix will be made available once the fix is complete.
    Best regards from Germany
    Stefan

    Hi Stefan,

     

    IBM has just released COBOL for AIX October 2015 PTF.  This fix pack contains fixes for the pipe FS39 issue. 

    I tried it with my existing app but have not been able to get it to work.  It seems the pipe i/o functionality has changed.

    Previously the reader and writer opens would both complete and the 2 programs would proceed. 

    Now the reader open waits for the writer to close the pipe.

     

    Jim

     

  • Stefan_Heublein
    Stefan_Heublein
    7 Posts

    Re: Named Pipes vs. IBM COBOL for AIX V4.1.1

    ‏2015-11-04T15:35:03Z  
    • texjim
    • ‏2015-11-03T20:44:29Z

    Hi Stefan,

     

    IBM has just released COBOL for AIX October 2015 PTF.  This fix pack contains fixes for the pipe FS39 issue. 

    I tried it with my existing app but have not been able to get it to work.  It seems the pipe i/o functionality has changed.

    Previously the reader and writer opens would both complete and the 2 programs would proceed. 

    Now the reader open waits for the writer to close the pipe.

     

    Jim

     

    Hi Jim,

     

    We downloaded and installed the October 2015 PTF for COBOL for AIX, V4.1.1.13, too.


    My tests of the named pipe functionality required by us were successful.
    So basically the example program and test case (mkfifo mypipe; cat mypipe & ./PIPEPGM) mentioned at the beginning of this discussion now works fine.

     

    Best regards from Germany
    Stefan