Topic
8 replies Latest Post - ‏2012-07-17T20:15:23Z by Erwin_Karbasi
wiedela
wiedela
19 Posts
ACCEPTED ANSWER

Pinned topic Agents and OSGi

‏2012-06-27T21:44:26Z |
Scale Experts
To use map reduce agents within the grid executing as OSGi bundles, you must put them into a bundle fragment. Once that is done, how do you execute the agents? I'm assuming that somehow you import the fragment into a client bundle and execute, but we have not been able to get that to work.

Finally, what is the best design pattern for accomplishing this? Do you want to make an agent factory and register it as a service?

Thanks
Updated on 2012-07-17T20:15:23Z at 2012-07-17T20:15:23Z by Erwin_Karbasi
  • andymc_
    andymc_
    16 Posts
    ACCEPTED ANSWER

    Re: Agents and OSGi

    ‏2012-06-28T16:12:41Z  in response to wiedela
    Hi Adam,

    Unlike other XS plug-ins, agents cannot be consumed by XS via OSGi services. They must be loaded directly by the XS runtime. The reason for this is that XS uses Java serialization (and de-serialization) to transfer the object to/from the remote server(s). As such, there is some complexity in ensuring that the agent class can be loaded by the XS runtime.

    Since it must be loaded directly, perhaps the easiest solution is to include the agent class(es) in a bundle fragment that attaches to the objectgrid.jar bundle. By attaching to the objectgrid.jar, it is essentially expanding the classpath of the bundle. The fragment can import additional packages needed by the agent - but these imports must be resolve-able - otherwise, the fragment will not attach to the host bundle. Likewise, the fragment can export packages that other parts of the application may need.

    Here is an example of how to execute an agent:
    http://publib.boulder.ibm.com/infocenter/wxsinfo/v7r1m1/topic/com.ibm.websphere.extremescale.doc/cxsdgapiex.html

    Note that the client code that executes the agent must import XS client APIs as well as the agent itself -- so it is critical the imports and exports between the client and agent are resolved (or else co-located in the same bundle fragment).

    Also note that the fragment must attach to the XS client bundle (ogclient.jar) on the client side as well as the XS server bundle (objectgrid.jar) - if you are using both XS bundles, then this requires two different fragments; one to attach to the client and one for the server. I have not personally tried this, but I suspect a more maintainable solution would be to create two manifest-only fragments that imports the agent package(s) and then put the agent code into a single bundle that exports its package(s) - then that bundle could be deployed in both the server and client. Alternatively, you could just use objectgrid.jar for both the client and server, and do everything from a single fragment.

    I realize that using agents in OSGi is somewhat complex. We are exploring ways to simplify this in the future.

    Hope this helps - if you have other questions or concerns, please let me know.

    Andy
    • Erwin_Karbasi
      Erwin_Karbasi
      186 Posts
      ACCEPTED ANSWER

      Re: Agents and OSGi

      ‏2012-07-01T15:18:28Z  in response to andymc_
      Hello Andy,

      We have the same issue when we are trying to run pre-load agent in the OSGI enabled grid server.

      I'd appreciate if you could provide a bundle fragment that illustrate this solution.

      Your direction would be highly appreciated.

      Thank you in advance,
      Erwin
      • andymc_
        andymc_
        16 Posts
        ACCEPTED ANSWER

        Re: Agents and OSGi

        ‏2012-07-03T18:27:37Z  in response to Erwin_Karbasi
        Hi Adam and Erwin,

        I am posting a slightly-modified example of one or our unit tests for agents in Liberty. Please note that agents in Liberty work slightly differently than in a vanilla-OSGi environment. In our XS 8.5 for Liberty implementation, EBAs are not supported -- you must install the bundles into the "grids" directory. Also, you only need to export your agent packages -- you do not need to package them in a bundle fragment. Aside from these differences, agents work the same in Liberty as they do in a vanilla-OSGi environment.

        This sample is fairly simple, in that it only returns server properties from the target server. Source code for the client is available in the agent client bundle.

        Instructions for running this agent sample in XS 8.5 for Liberty:
        0) Install WebSphere Liberty Profile 8.5 and WXS 8.5.0.1 (you can use either the Install Manager tool to install the product, or just use the archive installer).
        1) Unzip the attached agentSample.zip on top of your wlp directory.
        2) Run the agentSampleServer (i.e. wlp/bin/server run agentSampleServer ) and wait for it to finish starting up.
        3) Open a new command window and run the agentSampleClient (i.e. wlp/bin/server run agentSampleClient ).
        4) Open a new command window and telnet into the agentSampleClient's OSGi console and install and start the agent bundle and the agent client bundle, like so:
        $ telnet localhost 5679
        Trying 127.0.0.1...
        Connected to localhost.
        Escape character is '^]'.

        osgi> install file:///path/to/wlp/usr/servers/agentSampleClient/grids/com.ibm.ws.xs.agent.sample.jar
        Bundle id is 91

        osgi> install file:///path/to/wlp/usr/servers/agentSampleClient/grids/com.ibm.ws.xs.agent.client.sample.jar
        Bundle id is 92

        osgi> start 91

        osgi> start 92

        5) Review the output in the client and server windows to ensure that the agent executed as expected.
        Sample output from the server:

        $ ./server run agentSampleServer
        Launching agentSampleServer (wlp-1.0.0.20120419-1220/websphere-kernel_1.0.0) on IBM J9 VM, version pxa6460sr9fp2-20110625_01 (SR9 FP2) (en_US)
        AUDIT CWWKE0001I: The server agentSampleServer has been launched.
        Listening on port localhost/127.0.0.1:5678 ...
        WARNING CWOBJ0070W: The IBM implementation of the JVM is using a garbage collection policy that might affect performance.
        AUDIT CWWKZ0058I: Monitoring dropins for applications.
        AUDIT CWWKF0011I: The server agentSampleServer is ready to run a smarter planet.
        ServerPropsAgent putting agentSampleServer.server ={listenerPort=62123, catalogServiceEndPoints=localhost:62123, JMXServicePort=1099, serverName=agentSampleServer}
        ServerPropsAgent putting agentSampleServer.catalog ={catalogClusterEndPoints=agentSampleServer:localhost:6601:6611, isCatalog=true, domainName=myDomain, placementDeferralInterval=17000, enableQuorum=true, heartBeatFrequencyLevel=1}

        The last two lines indicate that the agent has successfully executed on the server.

        Sample output from the client:

        $ ./server run agentSampleClient
        Launching agentSampleClient (wlp-1.0.0.20120419-1220/websphere-kernel_1.0.0) on IBM J9 VM, version pxa6460sr9fp2-20110625_01 (SR9 FP2) (en_US)
        AUDIT CWWKE0001I: The server agentSampleClient has been launched.
        Listening on port localhost/127.0.0.1:5679 ...
        AUDIT CWWKZ0058I: Monitoring dropins for applications.
        AUDIT CWWKF0011I: The server agentSampleClient is ready to run a smarter planet.
        Activator - entering start() - com.ibm.ws.xs.agent.client.sample
        agentSampleServer.catalog = {catalogClusterEndPoints=agentSampleServer:localhost:6601:6611, isCatalog=true, domainName=myDomain, enableQuorum=true, placementDeferralInterval=17000, heartBeatFrequencyLevel=1}
        agentSampleServer.server = {listenerPort=62123, JMXServicePort=1099, catalogServiceEndPoints=localhost:62123, serverName=agentSampleServer}
        Activator - exiting start() - com.ibm.ws.xs.agent.client.sample

        The last three lines indicate that the agent successfully executed from the client and provided the requested results.

        I have not tested this in a vanilla-OSGi environment, but the only difference is that the agent classes will need to be in a bundle fragment (attached to the objectgrid.jar or ogclient.jar bundle).

        Hope this helps - if you run into problems or have more questions, please let me know.

        Thanks,

        Andy
        • Erwin_Karbasi
          Erwin_Karbasi
          186 Posts
          ACCEPTED ANSWER

          Re: Agents and OSGi

          ‏2012-07-04T12:21:33Z  in response to andymc_
          Hello Andy,

          Thank you a lot for sample and clarification.
          We have already succeed to build bundle fragment for our vanilla OSGI environment. Our fragment extends the objectGrid.jar with Agent classes.

          Thanks,
          Erwin
          • Erwin_Karbasi
            Erwin_Karbasi
            186 Posts
            ACCEPTED ANSWER

            Re: Agents and OSGi

            ‏2012-07-15T09:55:34Z  in response to Erwin_Karbasi
            Hello Andy,

            Do you have any assessment for Agent as a OSGI enabled?

            Thank you in advance,
            Erwin
            • andymc_
              andymc_
              16 Posts
              ACCEPTED ANSWER

              Re: Agents and OSGi

              ‏2012-07-17T14:12:08Z  in response to Erwin_Karbasi
              Hi Erwin,

              Are you asking if the product could handle agents as an OSGi service? This is something we've discussed casually (we don't have any formal plans in this space).

              Our 7.1.1 implementation (using bundle fragments to package user agent code) has been improved upon in 8.5 with Liberty (user agent code can exist in its own bundle as long as it exports the required agen packages).

              The main limitation to a full OSGi service-based solution is Java serialization. One of the primary values of agents is that the client can create them with different state which is serialized and then de-serialized on the server. The de-serialization process is not to friendly in OSGi - the de-serializing code (inside the WXS bundles) must be able to load the class(es) being de-serialized.

              If I am misunderstanding your question - or if you have another idea for this, please let me know.

              Thanks,

              Andy
              • Erwin_Karbasi
                Erwin_Karbasi
                186 Posts
                ACCEPTED ANSWER

                Re: Agents and OSGi

                ‏2012-07-17T20:15:23Z  in response to andymc_
                Hello Andy,

                You have understood perfectly my question and as usual you helped me a lot.

                Thanks,
                Erwin
  • wiedela
    wiedela
    19 Posts
    ACCEPTED ANSWER

    Re: Agents and OSGi

    ‏2012-06-28T17:37:30Z  in response to wiedela
    Hey Andy,
    Thank you for responding with your description, it wasn't clear to us how we were going to use all of our agents with XS.

    We did figure out how to create a bundle fragment with agents. One problem we have been having is if you use the OSGi Framework launcher included in Eclipse, the install order can change. What we figured out is that since we are starting the catalog and all of our containers programmatically, that if the fragment is installed AFTER the catalog and container bundles, it doesn't show up on the ORB. We couldn't figure out how to control the install order with the launcher, so we decided to go down the product path. Eventually, we need to execute this with a command line, so a product configuration file will work well for us.

    In regards to agents from clients, we are going to try to use a Blueprint file within the fragment to define factories to build out services. The agents will implement interfaces within a common bundle that can be shared. We do like the idea of a service factory building agents, so I'm hoping it works. If you have heard of anyone else doing this, we would like to hear about it.

    Thanks