Topic
  • 12 replies
  • Latest Post - ‏2014-01-23T14:55:31Z by yairbar
yairbar
yairbar
20 Posts

Pinned topic Deployment and Versioning prob in CICS 5.1 JVMServers:

‏2013-10-17T11:20:35Z |

It looks to me that CICS -OSGi Bundles dont give a real "new copy" ablity within a multi-package Java application. I.e., Disabling & Discarding a Bundle and replacing it with a new version one is NOT enough for causing
the new version of a class to be implemented - unless you recycle the JVMserver. It looks to me that an "old" class in NOT cleared within a JVM Server after its wrapping Bundle is Discarded - unless it belongs to the "upper" package of an appl.
Concluded that aft a long line of experiments. In most times I just had to recycle JVMserver to achieve "new copy".... am looking for a CONSISTENT Method of controlling life cycle of packages.
Does anybody like to comment ?
Thanks,
Yair
  • IvanHargreaves
    IvanHargreaves
    27 Posts

    Re: Deployment and Versioning prob in CICS 5.1 JVMServers:

    ‏2013-10-18T09:19:31Z  

    Hi Yair,

    Your observation is correct. The behaviour you see is standard OSGi behaviour. Once a bundle has reached the 'resolved' state, its dependencies are cached by the OSGi framework to ensure it always succeeds at runtime. If you wish to 'refresh' or use a newer version of a bundle (and your bundle interactions are done by calling packages directly), you must install a new version into the framework (with appropriate version increase in manifest) and then ensure the calling bundle targets the new version.

    Typically, the calling bundle will work with a range of versions. To explicitly use a newer version you must ensure your calling bundle no longer targets the lower version in its manifest, if the manifest does not specify a version, then the bundle is wired to any available version (which will be the older version in your case). The OSGi framework sees no reason to actively look for a newer implementation.

    You could of course use the Framework's refreshPackages method, which will cause the Bundle to be rewired, however this can potentially be disruptive (depending on your dependency tree).

    Beyond that, the usual advice is not to architect your application using packages directly but to use the much more dynamic micro-services (OSGi services). Services can come and go, and be replaced by newer implementations on the fly. Use Declarative Services or Blueprint to achieve this dynamism.

    Hope that helps,

    Ivan

  • yairbar
    yairbar
    20 Posts

    Re: Deployment and Versioning prob in CICS 5.1 JVMServers:

    ‏2013-10-23T07:15:51Z  

    Hi Yair,

    Your observation is correct. The behaviour you see is standard OSGi behaviour. Once a bundle has reached the 'resolved' state, its dependencies are cached by the OSGi framework to ensure it always succeeds at runtime. If you wish to 'refresh' or use a newer version of a bundle (and your bundle interactions are done by calling packages directly), you must install a new version into the framework (with appropriate version increase in manifest) and then ensure the calling bundle targets the new version.

    Typically, the calling bundle will work with a range of versions. To explicitly use a newer version you must ensure your calling bundle no longer targets the lower version in its manifest, if the manifest does not specify a version, then the bundle is wired to any available version (which will be the older version in your case). The OSGi framework sees no reason to actively look for a newer implementation.

    You could of course use the Framework's refreshPackages method, which will cause the Bundle to be rewired, however this can potentially be disruptive (depending on your dependency tree).

    Beyond that, the usual advice is not to architect your application using packages directly but to use the much more dynamic micro-services (OSGi services). Services can come and go, and be replaced by newer implementations on the fly. Use Declarative Services or Blueprint to achieve this dynamism.

    Hope that helps,

    Ivan

    Hello Ivan,

    Good to hear you again !!

    I am still at the packages methodology  (dont skilled myself yet in Blueprint or DS)...

    My test project  involves a Servlet (Managed as an Eclipse "Dynamic WEB Project) ,  which  uses("calls") a Business logic  Package's objects (Managed as an Eclipse Plug-in(OSGi) Project).  

    Each of them has a corresponding Bundle (mapped to a Cics Bundle). 

    Actually the first one's Bundle is composed of the WEB's bundle (warbundle) and another OSGi bundle which is called directly from it.

    The 2nd one is a regular OSGi .   This is the one I wish to be 'refreshable', i.e. to cause it to run in a new version without recycling JVMServer. 

    My problem is how to simulate a  "calling package" Manifest for declaring each time  the "Imported pcakage " excat version , where the WEB Project is not  actually an OSGi one.  I've tried to manipulate the Manifest of the Plug-in Proj which is bundled  together with it but it doesnt help until recyclling JVMServer. I've also tried (desparately) doing the same for the WEBcontent Manifes but it didnt help either. 

    Thanks for any advice,

     

    Yair

     

     

     

     

     

  • IvanHargreaves
    IvanHargreaves
    27 Posts

    Re: Deployment and Versioning prob in CICS 5.1 JVMServers:

    ‏2013-10-23T09:16:19Z  
    • yairbar
    • ‏2013-10-23T07:15:51Z

    Hello Ivan,

    Good to hear you again !!

    I am still at the packages methodology  (dont skilled myself yet in Blueprint or DS)...

    My test project  involves a Servlet (Managed as an Eclipse "Dynamic WEB Project) ,  which  uses("calls") a Business logic  Package's objects (Managed as an Eclipse Plug-in(OSGi) Project).  

    Each of them has a corresponding Bundle (mapped to a Cics Bundle). 

    Actually the first one's Bundle is composed of the WEB's bundle (warbundle) and another OSGi bundle which is called directly from it.

    The 2nd one is a regular OSGi .   This is the one I wish to be 'refreshable', i.e. to cause it to run in a new version without recycling JVMServer. 

    My problem is how to simulate a  "calling package" Manifest for declaring each time  the "Imported pcakage " excat version , where the WEB Project is not  actually an OSGi one.  I've tried to manipulate the Manifest of the Plug-in Proj which is bundled  together with it but it doesnt help until recyclling JVMServer. I've also tried (desparately) doing the same for the WEBcontent Manifes but it didnt help either. 

    Thanks for any advice,

     

    Yair

     

     

     

     

     

    Hi Yair,

    I'm presuming you have not applied CICS service (PM80214, or PM91667) yet, because if you had, you would not be able to call from a Dynamic Web Application (in the form of a WAR) to a CICS installed OSGi bundle. WebSphere closed that loophole from v8.5.0.1. The supported way to call from Web Applications to OSGi bundles is through the use of an EBA (Enterprise bundle archive) application which can be created and deployed in a CICS bundle. If you install the service, you will need the later version of CICS Explorer (v5.1.1) and the WebSphere Developer Tooling for Liberty (WDT v8.5.5) installed into your Eclipse (or you can use RAD or Rational Developer for z) if you want full support for WDT. See the CICS Infocenter for more details on the setup.

    That aside, the problem you face is that to call an OSGi bundle from a 'WAR' application requires the Web container to create a 'gateway' bundle to proxy your WAR application (because a WAR is not a versioned bundle, it is plain old classpath). Once the gateway bundle is built and it has wired to the specified packages (i.e. the second bundle is loaded onto its classloader). No amount of refreshing of the second bundle will allow the Web application to load the new version. This is because the WAR is not a bundle and does not respect or conform to OSGi 'versioning'. It is undesirable that the gateway bundle is not removed and refreshed in the OSGi framework when the application is reinstalled, however it is also not a supported mode of operation.

    I would strongly urge you to apply the CICS service, then create an "OSGi application project" (EBA) containing an "OSGi bundle project" (WAB) and any 'normal' OSGi bundles (or OSGi plugins) you require. The OSGi application project can be 'included' in a CICS bundle as before and exported to zFS. You drive the application in exactly the same way with the application's URI.

    The key differences are that a WAB or an OSGi bundle project with 'Web support' is the bundle aware form of a WAR. Including the OSGi bundles you require in the EBA creates an isolated, self-contained application. Or, if your OSGi bundles are common to more than one Web application, you can install them into Liberty's bundle repository instead of putting them in the EBA explicitly (see Application.MF of the OSGi application project).

    There are more details of how to do this in the latest refresh of the CICS Infocenter, but as usual we'll do our best to answer specific questions.

    thx,

    Ivan

  • yairbar
    yairbar
    20 Posts

    Re: Deployment and Versioning prob in CICS 5.1 JVMServers:

    ‏2013-10-23T13:14:16Z  

    Hi Yair,

    I'm presuming you have not applied CICS service (PM80214, or PM91667) yet, because if you had, you would not be able to call from a Dynamic Web Application (in the form of a WAR) to a CICS installed OSGi bundle. WebSphere closed that loophole from v8.5.0.1. The supported way to call from Web Applications to OSGi bundles is through the use of an EBA (Enterprise bundle archive) application which can be created and deployed in a CICS bundle. If you install the service, you will need the later version of CICS Explorer (v5.1.1) and the WebSphere Developer Tooling for Liberty (WDT v8.5.5) installed into your Eclipse (or you can use RAD or Rational Developer for z) if you want full support for WDT. See the CICS Infocenter for more details on the setup.

    That aside, the problem you face is that to call an OSGi bundle from a 'WAR' application requires the Web container to create a 'gateway' bundle to proxy your WAR application (because a WAR is not a versioned bundle, it is plain old classpath). Once the gateway bundle is built and it has wired to the specified packages (i.e. the second bundle is loaded onto its classloader). No amount of refreshing of the second bundle will allow the Web application to load the new version. This is because the WAR is not a bundle and does not respect or conform to OSGi 'versioning'. It is undesirable that the gateway bundle is not removed and refreshed in the OSGi framework when the application is reinstalled, however it is also not a supported mode of operation.

    I would strongly urge you to apply the CICS service, then create an "OSGi application project" (EBA) containing an "OSGi bundle project" (WAB) and any 'normal' OSGi bundles (or OSGi plugins) you require. The OSGi application project can be 'included' in a CICS bundle as before and exported to zFS. You drive the application in exactly the same way with the application's URI.

    The key differences are that a WAB or an OSGi bundle project with 'Web support' is the bundle aware form of a WAR. Including the OSGi bundles you require in the EBA creates an isolated, self-contained application. Or, if your OSGi bundles are common to more than one Web application, you can install them into Liberty's bundle repository instead of putting them in the EBA explicitly (see Application.MF of the OSGi application project).

    There are more details of how to do this in the latest refresh of the CICS Infocenter, but as usual we'll do our best to answer specific questions.

    thx,

    Ivan

    Ivan,

    Thanks a lot for this important & helpful   info , it saves us a vain effort under current version, and set new stuff for studying....

    I  assume  this fix is not deployed for  5.1 Developer Trial version ( we are still there...),  is it ??

    Thx again,

    Yair

     

  • IvanHargreaves
    IvanHargreaves
    27 Posts

    Re: Deployment and Versioning prob in CICS 5.1 JVMServers:

    ‏2013-10-23T13:25:54Z  
    • yairbar
    • ‏2013-10-23T13:14:16Z

    Ivan,

    Thanks a lot for this important & helpful   info , it saves us a vain effort under current version, and set new stuff for studying....

    I  assume  this fix is not deployed for  5.1 Developer Trial version ( we are still there...),  is it ??

    Thx again,

    Yair

     

    Hi Yair,

    I don't believe you can apply service to the Developer Trial. You can however get a newer version of the Developer Trial, and the latest version should included the updates to the Liberty Profile v8.5.0.1 along with CICS changes to support EBAs.

    :-)

    Ivan

  • yairbar
    yairbar
    20 Posts

    Re: Deployment and Versioning prob in CICS 5.1 JVMServers:

    ‏2014-01-13T13:37:00Z  

    Hi Yair,

    I don't believe you can apply service to the Developer Trial. You can however get a newer version of the Developer Trial, and the latest version should included the updates to the Liberty Profile v8.5.0.1 along with CICS changes to support EBAs.

    :-)

    Ivan

    Hello Ivan,

    OK, thanks.

    We've installed a new version of Developer Trial , and have used Eclipse 4.2.2 with CICS Explorer 5.1.1.

    We still have a problem when trying to run the EBA flavour of a dynamic WEB project -  which includes Servlet & JSP - that  calls a CICS/VSAM method which is  pacakaged within  a  separate  plug-in  project. 

    We are trying to implement a 'full OSGi' deployment of this application.

    So we have a CICS Bundle of an OSGi Application project (EBA) called "My1stEBA" -  which includes both - the previos WEB project as WAB (an OSGi bundle project called "JSP1WAB") , and the plug-in project called "VSAMtest1OSGi".

    Our problem is that when trying to run this EBA  from a browser's URL, a "Context Root Not Found" notification appears, and at the WLP server's messages.log there is an unclear  Java NPE error message, preceeded by a notification which identifies our EBA as "My1stEBA_2" (why ..._2"   ?? ) 

    Attached are Manifests of all componenets and the messages log.

    Appreciate any   help !!

     

    Regards,

    Yair

  • IvanHargreaves
    IvanHargreaves
    27 Posts

    Re: Deployment and Versioning prob in CICS 5.1 JVMServers:

    ‏2014-01-13T15:53:14Z  
    • yairbar
    • ‏2014-01-13T13:37:00Z

    Hello Ivan,

    OK, thanks.

    We've installed a new version of Developer Trial , and have used Eclipse 4.2.2 with CICS Explorer 5.1.1.

    We still have a problem when trying to run the EBA flavour of a dynamic WEB project -  which includes Servlet & JSP - that  calls a CICS/VSAM method which is  pacakaged within  a  separate  plug-in  project. 

    We are trying to implement a 'full OSGi' deployment of this application.

    So we have a CICS Bundle of an OSGi Application project (EBA) called "My1stEBA" -  which includes both - the previos WEB project as WAB (an OSGi bundle project called "JSP1WAB") , and the plug-in project called "VSAMtest1OSGi".

    Our problem is that when trying to run this EBA  from a browser's URL, a "Context Root Not Found" notification appears, and at the WLP server's messages.log there is an unclear  Java NPE error message, preceeded by a notification which identifies our EBA as "My1stEBA_2" (why ..._2"   ?? ) 

    Attached are Manifests of all componenets and the messages log.

    Appreciate any   help !!

     

    Regards,

    Yair

    Your general approach to creating and deploying the application looks correct.

    I think the reason the context-root isn't found is because the Web Application has not successfully installed. The reason the Web Application has not installed is given as an NPE. The diagnostics don't show what the cause of the NPE is, presumably some activator code issue, or there's a set-up conflict.

    Liberty is in control at the time of failure so CICS will not be able to provide further detail on the error, instead you'll need to turn Liberty tracing ON, which can be done with the following setting in your JVM profile:  -Dcom.ibm.ws.logging.trace.specification=*=all=enabled

    Or...there may already be diagnostics available in the FFDC directory of the Liberty server. For example WORK_DIR/<applid>/<JVMSERVER>/wlp/usr/servers/<Liberty server name>/logs.

    Reviewing the FFDC files should show the Java stacktrace that caused the error. If the NPE is in application code you should be able to identify and fix it yourself, otherwise, please post the failure files (and possible the trace.log created when Liberty trace is active) and I'll take a look.

    thx

    Ivan

  • yairbar
    yairbar
    20 Posts

    Re: Deployment and Versioning prob in CICS 5.1 JVMServers:

    ‏2014-01-14T13:23:59Z  

    Your general approach to creating and deploying the application looks correct.

    I think the reason the context-root isn't found is because the Web Application has not successfully installed. The reason the Web Application has not installed is given as an NPE. The diagnostics don't show what the cause of the NPE is, presumably some activator code issue, or there's a set-up conflict.

    Liberty is in control at the time of failure so CICS will not be able to provide further detail on the error, instead you'll need to turn Liberty tracing ON, which can be done with the following setting in your JVM profile:  -Dcom.ibm.ws.logging.trace.specification=*=all=enabled

    Or...there may already be diagnostics available in the FFDC directory of the Liberty server. For example WORK_DIR/<applid>/<JVMSERVER>/wlp/usr/servers/<Liberty server name>/logs.

    Reviewing the FFDC files should show the Java stacktrace that caused the error. If the NPE is in application code you should be able to identify and fix it yourself, otherwise, please post the failure files (and possible the trace.log created when Liberty trace is active) and I'll take a look.

    thx

    Ivan

    Hello Ivan,

    Tracing of Liberty is on, and I attach here the tracing file, may be it will help...

    FFDC directory has not been generated  (why ?)

    Meanwhile I'll try doublecheck setup as far as I can.

    Thanks again,

    Yair

     

    Attachments

  • IvanHargreaves
    IvanHargreaves
    27 Posts

    Re: Deployment and Versioning prob in CICS 5.1 JVMServers:

    ‏2014-01-14T16:09:18Z  
    • yairbar
    • ‏2014-01-14T13:23:59Z

    Hello Ivan,

    Tracing of Liberty is on, and I attach here the tracing file, may be it will help...

    FFDC directory has not been generated  (why ?)

    Meanwhile I'll try doublecheck setup as far as I can.

    Thanks again,

    Yair

     

    Hi Yair,

    I see there's an error during installation of the EBA.

    [1/13/14 12:34:35:064 GMT] 0000001a id=         org.apache.aries.subsystem.core.internal.StartAction         E resolve Unable to resolve bundles for subsystem/version/id My1stEBA/1.0.0/2: [VSAMtest1OSGi_1.0.0 [117], JSP1WAB_1.0.0 [119], org.osgi.service.subsystem.region.context.2_1.0.0 [116], org.eclipse.osgi_3.7.2.v20120110-1415 [118]]

    So I'm going to make a first guess at the problem. I think you are including the org.eclipse.osgi bundle (version 3.7.2) in your EBA application. This may be the problem because the version of Liberty shipped with CICS actually uses version 3.8.2 - and more importantly the EBA is therefore trying to install an OSGi framework implementation into an OSGi framework implementation, and I can't begin to second guess what would occur!

    So as a first step, you should not need to include that osgi jar in your EBA application. I suggest you remove it and redeploy the EBA. Let me know how that goes, hopefully it will solve the problem, but if not, delete any existing trace.log files, recreate the error, and post the new trace.log.

    thx

    Ivan

  • yairbar
    yairbar
    20 Posts

    Re: Deployment and Versioning prob in CICS 5.1 JVMServers:

    ‏2014-01-19T13:02:36Z  

    Hi Yair,

    I see there's an error during installation of the EBA.

    [1/13/14 12:34:35:064 GMT] 0000001a id=         org.apache.aries.subsystem.core.internal.StartAction         E resolve Unable to resolve bundles for subsystem/version/id My1stEBA/1.0.0/2: [VSAMtest1OSGi_1.0.0 [117], JSP1WAB_1.0.0 [119], org.osgi.service.subsystem.region.context.2_1.0.0 [116], org.eclipse.osgi_3.7.2.v20120110-1415 [118]]

    So I'm going to make a first guess at the problem. I think you are including the org.eclipse.osgi bundle (version 3.7.2) in your EBA application. This may be the problem because the version of Liberty shipped with CICS actually uses version 3.8.2 - and more importantly the EBA is therefore trying to install an OSGi framework implementation into an OSGi framework implementation, and I can't begin to second guess what would occur!

    So as a first step, you should not need to include that osgi jar in your EBA application. I suggest you remove it and redeploy the EBA. Let me know how that goes, hopefully it will solve the problem, but if not, delete any existing trace.log files, recreate the error, and post the new trace.log.

    thx

    Ivan

    Hello Ivan,

    Thanks.  The inclusion of  org.eclipse.osgi bundle was one of my desparate attempts to cope with the original problem. This one  was not  solved yet  after removing that stuff  as well (please  see below) but  your comments are mostly  important and educating as usual !!

    Current status is that a 'Servlet only' project  (called JSP1WAB,  using servlet YairJSP1WAB) is bundled as an EBA  and doesnt have any reference to another bundle - just for arranging a step by step test case.

    A "Context Root Not found" message appears at the browser returned page, and at the error logs an "illegatArgumentException" appears regarding an  "1.0.0." (invalid) version format. 

    Hence in the relevant manifests I fail to see this invalid version (they all mention "1.0.0"  w/out this dot  as suffix...)

    Attached relevant manifests, messages.log and trace.log.   Btw   FFDC were not generated.

    Hope you will find time to have a look.

    Thanks,

    Yair 

  • IvanHargreaves
    IvanHargreaves
    27 Posts

    Re: Deployment and Versioning prob in CICS 5.1 JVMServers:

    ‏2014-01-20T19:01:08Z  
    • yairbar
    • ‏2014-01-19T13:02:36Z

    Hello Ivan,

    Thanks.  The inclusion of  org.eclipse.osgi bundle was one of my desparate attempts to cope with the original problem. This one  was not  solved yet  after removing that stuff  as well (please  see below) but  your comments are mostly  important and educating as usual !!

    Current status is that a 'Servlet only' project  (called JSP1WAB,  using servlet YairJSP1WAB) is bundled as an EBA  and doesnt have any reference to another bundle - just for arranging a step by step test case.

    A "Context Root Not found" message appears at the browser returned page, and at the error logs an "illegatArgumentException" appears regarding an  "1.0.0." (invalid) version format. 

    Hence in the relevant manifests I fail to see this invalid version (they all mention "1.0.0"  w/out this dot  as suffix...)

    Attached relevant manifests, messages.log and trace.log.   Btw   FFDC were not generated.

    Hope you will find time to have a look.

    Thanks,

    Yair 

    I think the next problem is due to an extraneous full stop "." at the end of the version number of your application. This has caused an invalid format and IllegalArgumentException.

    [1/19/14 13:15:52:093 GMT] 0000001a id=         com.ibm.ws.app.manager.esa.internal.SubsystemHandler         A CWWKZ0403E: A management exception was generated when trying to install the application EBAforJSP1 into an OSGi framework.  The error text from the OSGi framework is: java.lang.IllegalArgumentException: invalid version "1.0.0.": invalid format
     

    Try correcting the version number and see where that gets us..

    thx, Ivan

  • yairbar
    yairbar
    20 Posts

    Re: Deployment and Versioning prob in CICS 5.1 JVMServers:

    ‏2014-01-23T14:55:31Z  

    I think the next problem is due to an extraneous full stop "." at the end of the version number of your application. This has caused an invalid format and IllegalArgumentException.

    [1/19/14 13:15:52:093 GMT] 0000001a id=         com.ibm.ws.app.manager.esa.internal.SubsystemHandler         A CWWKZ0403E: A management exception was generated when trying to install the application EBAforJSP1 into an OSGi framework.  The error text from the OSGi framework is: java.lang.IllegalArgumentException: invalid version "1.0.0.": invalid format
     

    Try correcting the version number and see where that gets us..

    thx, Ivan

    Thanks, Ivan.

    Well , good news...

    ...Since I didnt find that  extranous full full stop in any  manifest, I've regenerated projects.

    And in order to run Servlet successfully I had also to double check that:

     

    1.Active Target platform in Eclipse is "CICS TS V5.1 With Liberty"

    2. WAB project is built with WEB Support

    3. Servlet name  (i.e. not the project name)  is specified at "WEB-Context"   def in the WAB manifest.

    Then the EBA  proj which contains that WAB was packed in CICS Bundle which was deployed to CICS, and Servlet was successfully run from Browser using URL with its name.

    At this stage I have enlarged this EBA  to include also  an OSGi project's Bundle of CICS/VSAM calls which are referrred to from the WAB project's Servlet.  A CICS Bundle includes both the ebabundle AND the osgibundle and deployed to CICS.

    I am HAPPILY  able now to change logic in either the Servlet or the VSAM class or both, and "NewCopy"ing it  just  by Disable/Enable the CICS bundle (aft re-deployment),  using a current running  JVM,   no prob.

    This basic step using this new version  looks OK and I thank you very much !!!

    (i'll try going forward e.g.  with 'shared bundles', and with  more  complicated versioning,   etc...)

     

    Thanks,

    Yair Barzilai