Topic
IC4NOTICE: developerWorks Community will be offline May 29-30, 2015 while we upgrade to the latest version of IBM Connections. For more information, read our upgrade FAQ.
No replies
PeteInOxford
PeteInOxford
25 Posts
ACCEPTED ANSWER

Pinned topic Persistent Working-Storage under IMS?

‏2013-09-11T08:26:51Z |

Ambling round a client's IMS MPP regions the other day I came across a program, essentially an I/O module, that very clearly expects Working-Storage(W-S) to be persistent across tasks (it caches some very high activity data in W-S). Unfortunately W-S isn't persistent any more!  I've had a hunt through the documentation and it seems that this was fairly easily achieved using OS/VS COBOL (preload module, NOENDJOB option etc.) and COBOL/2 (LRR runtime option and so forth).

There's almost no mention of a similar capability being available for Enterprise COBOL, though, which seems (to put it mildly) a crying shame - particularly as we now operate in a storage-rich environment. I note that RTEREUS is "not recommended for use under IMS" but no reason is given for that.

So I suppose the crux of the matter is: If support for persistent working-storage is still there, how do I do it; if persistent working-storage is a thing of the past, is there a recognized / recommended alternative technique? For obvious reasons I'd like to keep source-level changes to a minimum, but I'm not above changing things around if I must as this is costing a great deal of money.

I'd greatly appreciate comment on this - working-storage persistence seems so very useful a capability that it's outright bizarre for it to have disappeared. 

Thanks in advance!

Pete. 

 

 

 

   

Updated on 2013-09-20T11:26:54Z at 2013-09-20T11:26:54Z by PeteInOxford
  • BillWoodger
    BillWoodger
    126 Posts
    ACCEPTED ANSWER

    Re: Persistent Working-Storage under IMS?

    ‏2013-09-17T23:13:15Z  in response to PeteInOxford

    OK, I know nothing about IMS :-)

    But this question is a little lonely...

    There are some IMS-knowledgeable people at www.ibmmainframes.com.

    My blind-as-a-bat suggestion would be to acquire a lump of storage that does not get destroyed, get a pointer to that storage to the IO routine, and use that storage as the "persistent" storage.

    I'm guessing you have thought of something like that, but you need to know if it is a "good under IMS" solution...

  • wmklein
    wmklein
    79 Posts
    ACCEPTED ANSWER

    Re: Persistent Working-Storage under IMS?

    ‏2013-09-18T14:44:01Z  in response to PeteInOxford

    Working-Storage across "tasks" (and across run-units) is difficult to achieve.

    I can imagine (at least) 3 ways to achieve this (I don't have experience with these).

    1) Use something like a linear dataset (and Linkage Section) to store and retrieve data.   However, if  you are doing simultaneous reads AND write, there might well be synchronization problems

    2) IBM does support multi-tasking in COBOL under IMS. See for example

     http://pic.dhe.ibm.com/infocenter/pdthelp/v1r1/index.jsp?topic=%2Fcom.ibm.entcobol.doc_3.4%2Ftpthr02.htm

    Therefore, it SHOULD be possible to have a non-COBOL (Assembler) driver)( that actually starts each task.  The WS data would be persistent while the Local-Storage data would be for each instance of the program.   However, if you are talking about a common subroutine that is called by multiple  programs and transactions, I don't think this would for that.

    3) Probably the "cleanest" way to do this would be to place the data that you want into an IMS (or DB2) database and then access it via normal means - both for reading and updating.  In that way standard IMS (facilities would handle the synchronization issues

     

  • infocat
    infocat
    8 Posts
    ACCEPTED ANSWER

    Re: Persistent Working-Storage under IMS?

    ‏2013-09-18T17:23:57Z  in response to PeteInOxford

    Can you clarify something for me?  Are you saying that this used to work (under older version of COBOL), but it no longer works (with Enterprise COBOL)?  Or are you saying you think you could have gotten it to work with older versions of COBOL but you can't see how to make it work with Enterprise COBOL?  Because unless your MPP is a long running 'task' that handles multiple transactions (I don't know how IMS MPPs work) I don't see how you could possibly, within just COBOL "memory management" itself, get this to work.  As Bill says above, you'd have to save off the WS area to some persistant storage, and then retrieve it for the new transaction/task.

    But in the end its not clear to me what you are trying to accomplish, as there (as far as I know) has never been support for persisting working-storage outside of a "COBOL run unit".  I've never heard of NOENDJOB or LRR, but they appear to me to refer to the COBOL runtime and as far as I can tell have nothing to do with COBOL working-storage. 

    Just my thoughts...

    Frank

  • BillWoodger
    BillWoodger
    126 Posts
    ACCEPTED ANSWER

    Re: Persistent Working-Storage under IMS?

    ‏2013-09-19T15:12:50Z  in response to PeteInOxford

    "[A] program, essentially an I/O module ...  caches some very high activity data in W-S".

     

    I saw this as a program which is called from many places to get information. The most common information is stored in W-S (perhaps originally coming from a file/db), and returned from there, otherwise the program goes off to access the file/db. A performance thing. It is all very well having something in a "buffer" somewhere so there is no physical IO, but it still occupies time, there and back, to discover that if you go through the normal file/db route and rely on it being in the buffer.

    I assume that there is information that is not/connot be "hard coded" in the W-S (if it could, problem goes away).

    I don't think storing the data on any file or database would serve the performance issue.

    With RTEREUS a "reuable run-time environment" can be made for a COBOL program . For instance, try a COBOL Compiler exit without RTEREUS and prepare to go for lunch whilst the compile is running and the COBOL exit is constantly called/cancelled. Having made a COBOL program persistent through RTEREUS, then the W-S is persistent. But Pete points out that is is not recommended for IMS.

    Perhaps I've misunderstood.

    Updated on 2013-09-19T15:13:43Z at 2013-09-19T15:13:43Z by BillWoodger
  • wmklein
    wmklein
    79 Posts
    ACCEPTED ANSWER

    Re: Persistent Working-Storage under IMS?

    ‏2013-09-19T19:00:39Z  in response to PeteInOxford

    RTEREUS is (ws) intended to keep the COBOL run-time environment available.  This was intended to avoid going thru the enire run-time initialization process every time a COBOL main program was started.

    At NOT time was it documented as keeping "program-specific data" persistant across run units.

    The same could be said for the OS/VS COBOL NOENDJO process.

    There is a "world of difference" between persistence of application data vs persistence of the COBOL run-time environment.

    Having said this and knowing (somewhat)  how OS/VS COBOL used to work.  I don't remember ever doing it, but using NOENDJOB with an IMS preloaded program might actually have done what was reported.  I can almost guarantee that it was never documented as doing this.

    Similarly, I could imagine that using a NORENT Enterprise COBOL program that was preloaded (IMS preload) there might be some way of tricky the system into keeping WS values across run-units.  I don't know how, and I am certain it wouldn't be supported.In fact, I just checked and RENT is required for IBM preload, so there really, REALLY, would be no way to keep WS across run-units.

    As far as I remember, an IMS regions doesn't have anything similar to a CICS CWA - a place to store region-wide data. It it did, then that would be a possible solution to this issue too.

    • PeteInOxford
      PeteInOxford
      25 Posts
      ACCEPTED ANSWER

      Re: Persistent Working-Storage under IMS?

      ‏2013-09-20T11:22:29Z  in response to wmklein

      Gentlemen!

       

      Thank you muchly for the replies - they _are_ appreciated. Here's the non-existent documentation, Mr. Klein (I don't know how you prefer to be addressed - apologies if I've got it wrong):

      http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/IGYMG202/APPENDIX1.13.3.2?SHELF=&DT=20001004155832&CASE=

       

      The salient bit is as follows:

       

      In the VS COBOL II run-time environment when using the LIBKEEP run-time option, OS/VS COBOL programs compiled with ENDJOB were treated as NOENDJOB. NOENDJOB was required for preloaded OS/VS COBOL programs.

      Because IBM COBOL programs and their resources (all work areas and library routines) are always deleted at termination, the way ENDJOB and NOENDJOB work in the Language Environment run-time environment allows both IBM COBOL and OS/VS COBOL programs to be treated similarly with respect to last-used state.
      

      In the OS/VS COBOL run-time environment, ENDJOB was the strongly recommended compiler option when executing IMS transactions invoking nonpreloaded COBOL programs.

      In the VS COBOL II run-time environment when using the LIBKEEP run-time option, OS/VS COBOL programs compiled with ENDJOB were treated as NOENDJOB. NOENDJOB was required for preloaded OS/VS COBOL programs.

      >>> Note: When using LRR, NOENDJOB behavior is always in effect. <<<

       

      Now, this is old - very old. But the functionality used to be there (I found other references too).

       

      As for other things: update frequency for the table in question is measured in updates per decade - it is rock steady. The MPP regions in question are recycled daily, so no issues there. A colleague and I looked into this a bit and the COBOL runtime seems to recognize that a program is already part of the run unit if the entry point address didn't change - so might marking the module for preload do the job?

       

      Bill (Woodger): You've hit the nail precisely on the head. Almost any other method of storing the data would be more expensive in terms of both CPU and - more significantly - programmer time than just hanging on to working-storage, as there are a goodly number of places where an idiotically-simple-to-implement  technique (bang it in the preload list) would save just *oodles* of time - at present the database is being hit with an absolutely obscene frequency. This offends my sense of propriety :-)

      Marking the module for preload seems at least worth a try, so I have requested that a test be run.  

      Seems very odd that a region-wide solution isn't available (I mean, I only need one - one! - fullword and I can do the rest myself). There is the IMS Scratch Pad Area (SPA) but it looks tricky to use so I don't want anything to do with it for the reasons outlined above.

      Thanks again for the replies - I really am grateful. And this wandering round the (bleeding?) edge of the COBOL world is fun, too. 

       

      Pete.

      And this may be it (next topic from the one above). Does LRR apply under IMS these days?

      APPENDIX1.13.3.3 When programs remain in the last-used state

       

      A program remains in last-used state between COBOL transactions only when all the following conditions are true:

      • It is an OS/VS COBOL program compiled with the NOENDJOB option or LRR is being used.
      • It has been link-edited with the REUS option.
      • It has been preloaded.
      • It is a dynamically called subprogram or a statically called subprogram that has, itself, been called by a dynamically called subprogram.

         

       

       

       

      Updated on 2013-09-20T11:27:50Z at 2013-09-20T11:27:50Z by PeteInOxford
  • wmklein
    wmklein
    79 Posts
    ACCEPTED ANSWER

    Re: Persistent Working-Storage under IMS?

    ‏2013-09-20T13:33:49Z  in response to PeteInOxford

    Did you see the paragraph that says,

     "Because Enterprise COBOL programs and their resources (all work areas and library routines) are always deleted at termination, the way ENDJOB and NOENDJOB work in the Language Environment runtime environment allows both Enterprise COBOL and OS/VS COBOL programs to be treated similarly with respect to last used state."

    In other words, it isn't (exactly) Enterprise COBOL that is giving you "bad" results.  It is LE itself.  If you run your existing OS/VS COBOL, NOENDJOB program under LE it will only get "persistent" last-used state in with the restricitons you listed above.

    EVERY other COBOL program (OS/VS COBOL or later) will be entered in initial state (the first time it is called in a run-unit)

    I know that for performance reasons, you want a different answer, but you aren't going to find one.  (This is especially true for programs compiled with RENT)

    You must use some "external" facility (file, database, SPA, "monk with a quill pen" - whatever) to communicate between run-units .

    Of course, there is nothing stopping you from keeping this routine,

    • compiled with OS/VS COBOL (and NOENDJOB)
    • preloaded
    • link-edited with REUS

    You will need to call in DYNAMICALLY (which has some overhead) and your CALLING program will need to use DATA(24) if you want to pass data to the subroutine.

    You also need to be aware that starting with z/OS V2 (GA next week), you will get run-time warning messages for every OS/VS COBOL program that you call.  (Sort-of hints that OS/VS COBOL run-time support MAY be ended during the life of z/OS V2).

    Of course, if you ever want to upgrde to Enterprise COBOL V5R1 (or lter), using an OS/VS COBL subroutine won't work.  Starting with this release, you can't have ANY mixture of Enterprise COBOL 5.1 and OS/VS COBL programs - even if they are dynamically Called.

    If you are running OS/VS COBOL preloaded programs in IMS today, you may want to searchthe web for problems with an OLD user modification to the IMS routine DFSPCC20

    Updated on 2013-09-20T13:41:45Z at 2013-09-20T13:41:45Z by wmklein
    • PeteInOxford
      PeteInOxford
      25 Posts
      ACCEPTED ANSWER

      Re: Persistent Working-Storage under IMS?

      ‏2013-09-20T15:41:21Z  in response to wmklein

      Um, well, yes, as a matter of fact I did see that. Let's both read it again:

      A program remains in last-used state between COBOL transactions only when all the following conditions are true:

      • It is an OS/VS COBOL program compiled with the NOENDJOB option or LRR is being used.
      • It has been link-edited with the REUS option.
      • It has been preloaded.
      • It is a dynamically called subprogram or a statically called subprogram that has, itself, been called by a dynamically called subprogram.

      Does my interest in LRR (or any functional equivalent under LE) now become clear? LRR is Library Routine Retention, which is a very slightly obtuse COBOL/2 runtime option and nothing to do with OS/VS COBOL at all.  And as I said in my original post, it is very unlike IBM to remove functionality completely - especially when it's so very, very useful (and thus likely to be widely used).

      If it helps, the client has no OS/VS COBOL (and I wouldn't recommend going back if you pointed a gun at my head). I can be sure of that  'coz I ran an analysis of the load libraries. They compile with DYNAM, too. By definition the program I'm interested in, and any similar ones, are subprograms. As noted in an earlier post their load modules are all marked REUS (not RENT) - something I used to find odd in IMS shops. 

      It all hinges on whether LE runs in a functionally equivalent mode to LRR-enabled COBOL/2 under an IMS MPP. As I can't imagine why anyone would not use LRR-or-equivalent in those circumstances (to avoid all that expensive up-and-downing with the environment) I'd like to know. I really do think it's a perfectly reasonable question. 

      As ever, thanks for the reply. It all helps focus on the issue.   

  • wmklein
    wmklein
    79 Posts
    ACCEPTED ANSWER

    Re: Persistent Working-Storage under IMS?

    ‏2013-09-20T17:48:45Z  in response to PeteInOxford

    A couple of things

    1) Regarding "is LRR still supported", check out the Enterprise COBOL V5R1 (released this past June) statement at

     http://pic.dhe.ibm.com/infocenter/pdthelp/v1r1/topic/com.ibm.entcobol.doc_5.1/PGandLR/tasks/tpeff23.html

    "IMS: If your application runs under IMS, preloading the application program and the library routines can help reduce the overhead of loading and searching. It can also reduce the input-output activity.

    For better system performance, use the RENTcompiler option and preload the applications and library routines when possible. You can also use the Language Environment library routine retention (LRR) function to improve performance in IMS/TM regions"

    2) regarding "REUS" vs "RENT".  Don't confuse the RENT compiler option with the RENT linkage-editor (binder) attribute. The recommendation from IBM is to use the RENT compiler option (and this will default to setting the RENT linkage editor option as well). But with this compiler option in effect, there is simply NO WAY that WS could "persist" between run-units. (Before VS COBOL II, you had to use REUS because the compiler couldn't produce RENT object code)

    3) Again on LRR, this is a method of avoiding COBOL run-time library initialization.  It has NOTHING to do with maintaining "last used state" of application programs and data.  The two issues of "performance based on the need to reinitialize the COBOL/LE run-time" versus "keeping data across run-units are quite separate in post-OS/VS COBOL IMS applications.

    ***

    You are correct that "removing functionality" is medium unusual for IBM.  On the other hand, the ability to have persistent data across run-units was removed with VS COBOL II, R1 (circa 1987).  As far as I know, (and I have been involved with both GUIDE and SHARE requirements most of this time) NO ONE has ever asked IBM to re-introduce this facility.  (Lots of things went away with VS COBOL II, R1 - remember "READY/RESET TRACE" "EXHIBIT NAMED" Even TCAM communications section)

    My best guess is that most shops are "happy" to get the advantages of RENTobject code and to use IMS facilitiesd 9databases, spa, whatever) to share data across transactions.  On the other hand, CICS does provide the CWA (and some other features) to share data across transactions in the same region, so MAYBE sites that have a need to do this and don't want to work with IMS (and LE) the way they are have migrated to CICS.  (At one time, IBM actually had and shared statistics on sites moving from IMS /DC to CICS.  Today, I think it is more likely for sites to be moving from IMS/DC to web-based solutions.)

    Going back many steps in this thread, you might want to check out the IMS list-server to see what other sites do about this.  See

     http://www.lsoft.com/scripts/wl.exe?SL1=IMS-L&H=IMSLISTSERV.BMC.COM

    P.S.  I am a little confused about one thing.  Does the existing program actually do what you think it wants to do?  I think you just said that they don't have any OS/VS COBOL programs in their shop.  What language (compiler) is used for the existing program?  Does it actually use persistent data across transactions?  If not (but the program is designed to do so) what is happening ithe applications today?  Maybe the program was DESIGNED to use persistent data, but hasn't been doing so for how many years since they got rid of OS/VS COBOL.

     

    • PeteInOxford
      PeteInOxford
      25 Posts
      ACCEPTED ANSWER

      Re: Persistent Working-Storage under IMS?

      ‏2013-09-20T22:27:34Z  in response to wmklein

      P.S.  I am a little confused about one thing.  Does the existing program actually do what you think it wants to do?  I think you just said that they don't have any OS/VS COBOL programs in their shop.  What language (compiler) is used for the existing program?  Does it actually use persistent data across transactions?  If not (but the program is designed to do so) what is happening ithe applications today?  Maybe the program was DESIGNED to use persistent data, but hasn't been doing so for how many years since they got rid of OS/VS COBOL.

      Well, nearly that. Your last statement is, I think, pretty close to the mark. It's doing millions - no, on reflection, probably billions - of database fetches per hour it really doesn't need to because it reloads the W-S lookaside table every time it gets reinitialized, instead of remaining in storage as it expects. The PWFI facility is helping, a lot, but not enough. It just seemed (and seems) really really weird not to have a way of making IMS transaction data persist. Dropping the W-S table loads and just going directly to the (DB2) table would probably increase the I/O rate because of PWFI, but it's still much too high as matters stand.

      My interest in LRR stemmed from reading the statement above - i.e. that LRR was a prerequisite for working-storage retention. I'd run into it before and couldn't find it again until I remembered that the IBM Bookserver library isn't indexed by Google (which is a shame). But - and I'm afraid it's a big but - the statement quoted above comes from a later release of COBOL than COBOL/2 - COBOL for OS/390 & VM, in fact, a few releases later, and so it directly contradicts your assertion that  

      the ability to have persistent data across run-units was removed with VS COBOL II, R1 (circa 1987).    

      You see why I'm confused?

      To add to the confusion, I take your point about RENT versus REUS (and what they mean, and where they get specified). You are perfectly right and your argument makes perfect sense. But during initialization of the regions., the IMS MPPs display a message about having pre-loaded non-re-entrant programs. And IMS MPPs are single-threaded, in this instance anyway, so RENT probably makes no difference at the MPP level. Everything contradicts everything else.

      On a happier note, I do indeed remember the "lost" facilities of OS/VS COBOL very well. Many a time I've wished to have USE FOR DEBUGGING ON ALL REFERENCES back. The ability to pick off the SYNAD area for an I/O error was more than nice, too, if you did a lot of work with tapes. But I'm perfectly happy about never having had to code a SEEK - on IBM kit, anyway!

      Thanks, again, for the reply. I will go and make a nuisance of myself on the IMS mailing list you so kindly pointed to :-) 

      Pete.

       

      Updated on 2013-09-20T22:34:21Z at 2013-09-20T22:34:21Z by PeteInOxford
    • BillWoodger
      BillWoodger
      126 Posts
      ACCEPTED ANSWER

      Re: Persistent Working-Storage under IMS?

      ‏2013-09-20T23:02:24Z  in response to wmklein

      I think Pete covered the PS in his original post. The program is coded to work on the assumption that it has previous values from the W-S. It does't currently work as intended, but goes off to do whatever IO each time. I'd guess the "cacheing" is doing nothing currently but actualy slowing the process down :-)

       

  • PeteInOxford
    PeteInOxford
    25 Posts
    ACCEPTED ANSWER

    Re: Persistent Working-Storage under IMS?

    ‏2013-10-08T16:09:01Z  in response to PeteInOxford

    Gentlefolk,

    I thought it might perhaps be useful to others for me to document where this ended up. It turns out (verified by testing) that the IMS annexe to
    the documentation I quoted is correct in every detail. I think for all of us the reference to OS/VS COBOL rather clouds our judgement.

    The four criteria I quoted for W-S retention across COBOL transactions often all apply. Here's how to check that you can use this:

    •It is an OS/VS COBOL program compiled with the NOENDJOB option >>or<< LRR is being used.

    I doubt that first condition applies very frequently these days but LRR will be in use if module CEELRRIN is specified as an entry in the PREINIT
    modules list - member DFSINTxx if you specify PREINIT=xx in the usual startup JCL (it's the fourth positional parameter). I expect most places
    will have this on (as was very helpfully pointed out, performance can, not will, absolutely suck without it).

    We are seeing this working with Enterprise COBOL V4.2 so age isn't an issue (if only that were true in real life!)

    •It has been link-edited with the REUS option.

    I imagine compiling (not link-editing it) it as RENT would do no harm at all, but might not be mandatory - after all, it will be single-threaded
    inside an MPP (that's what REUS means, for that matter), won't it? Anyway, REUS it is.

    •It has been preloaded.

    That's the kicker. Preload the bugger. 

    •It is a dynamically called subprogram or a statically called subprogram that has, itself, been called by a dynamically called subprogram.

    Most people do dynamic calls these days. The overhead is there but it isn't that awful.

    And there we have it. Thanks to *all* who contributed, whether pro or anti, it all helped.

    Updated on 2013-10-08T16:10:28Z at 2013-10-08T16:10:28Z by PeteInOxford