Topic
2 replies Latest Post - ‏2012-08-07T22:57:18Z by lisaw
wiedela
wiedela
19 Posts
ACCEPTED ANSWER

Pinned topic Starting large amounts of containers with OSGi

‏2012-06-21T16:26:47Z |
Before I go down this development path, I wanted to ask the experts if what we are doing sounds like a good idea. We have the scenario of standing up large numbers of containers (>100) for our environment and we are using OSGi bundles for deployments. The issue we started having is how are we going to start all the containers within our environments with as little maintenance as possible.

Option 1: A couple of threads/articles talked about pointing the configuration areas to different Blueprint files. This was not the best idea for us because we were going to need to create ~100 different Blueprint files and corresponding configurations (bad maintenance tail).

Option 2: We were thinking how we could using the config.ini files for each container. This seemed like a better idea, but we run into the same maintenance tail as with the Blueprint files.

We came to the conclusion that we wanted to be able to pass a single/simple parameter (container ID derived from the OS) when we would start the containers. The development/integration path we are going to take is to start each container programmatically from a bundles Activator object. We would read in the server configuration file, change the container's ID based on the parameter, and start the container.

My questions are:
  • Is this a good idea?
  • Has anyone done this already?
  • What should we be concerned with implementing containers this way?
  • Are we completely lost? (don't be shy if you think we are, we are engineers and have no feelings)

We do feel like we are getting close to a strong stable implementation of XS with OSGi, but we are really in the infancy of understanding both technologies. Any and all help/guidance is greatly appreciated.

Thanks
Updated on 2012-08-07T22:57:18Z at 2012-08-07T22:57:18Z by lisaw
  • wiedela
    wiedela
    19 Posts
    ACCEPTED ANSWER

    Re: Starting large amounts of containers with OSGi

    ‏2012-06-22T15:24:21Z  in response to wiedela
    Got the catalog server and a couple of containers up and running, but I'm running into a couple of problems with using the grid.

    First off when I start a container, I get the following error twice in my SystemErr.log for the Catalog:

    6/22/12 9:12:58:457 MDT 00000018 SystemErr R java.lang.ClassNotFoundException: zero.core.context.GlobalContext
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at java.lang.Class.forName(Class.java:134)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.ws.xs.stats.StatsUtil.getXSAGridName(StatsUtil.java:572)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.ws.xs.stats.gridmap.injector.InjectorUtils.isInjectionEnabled(InjectorUtils.java:22)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.ws.xs.stats.gridmap.injector.ObjectGridConfigurationInjector.fireEvent(ObjectGridConfigurationInjector.java:46)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.ws.objectgrid.deployment.policy.DeploymentPolicyMapsetInjector.invokeListeners(DeploymentPolicyMapsetInjector.java:144)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.ws.objectgrid.deployment.policy.DeploymentPolicyMapsetInjector.getUpdatedDeploymentPolicy(DeploymentPolicyMapsetInjector.java:108)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.ws.objectgrid.catalog.placement.CatalogServiceImpl.registerContainerAndAugmentDeploymentPolicy(CatalogServiceImpl.java:924)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.ws.objectgrid.catalog.IDLPlacementServicePOA._invoke(IDLPlacementServicePOA.java:164)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.CORBA.poa.POAServerDelegate.dispatchToServant(POAServerDelegate.java:435)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.CORBA.poa.POAServerDelegate.internalDispatch(POAServerDelegate.java:362)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.CORBA.poa.POAServerDelegate.dispatch(POAServerDelegate.java:258)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.rmi.iiop.ORB.process(ORB.java:517)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.CORBA.iiop.ORB.process(ORB.java:1575)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.rmi.iiop.Connection.doRequestWork(Connection.java:2965)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.rmi.iiop.Connection.doWork(Connection.java:2848)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.rmi.iiop.WorkUnitImpl.doWork(WorkUnitImpl.java:64)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at com.ibm.ws.objectgrid.thread.ORBThreadPool$1.run(ORBThreadPool.java:82)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
    6/22/12 9:12:58:457 MDT 00000018 SystemErr R at java.lang.Thread.run(Thread.java:769)

    I built a simple ReduceGridAgent to execute, and I can't get it to execute on the grid. We are using OSGi bundles to run both the catalog server and containers, and even though I can start the bundle that contains the agent in both the catalog and container, no go:

    Exception in thread "P=38911:O=0:CT" com.ibm.websphere.objectgrid.ObjectGridRuntimeException: org.omg.CORBA.INTERNAL:
    An internal error happened processing the request vmcid: 0x0 minor code: 0 completed: No
    at com.ibm.ws.objectgrid.client.ORBClientCoreMessageHandler.sendMessage(ORBClientCoreMessageHandler.java:747)
    at com.ibm.ws.objectgrid.client.ORBClientCoreMessageHandler.sendMessage(ORBClientCoreMessageHandler.java:1243)
    at com.ibm.ws.objectgrid.client.RemoteTransactionCallbackImpl.sendMessage(RemoteTransactionCallbackImpl.java:710)
    at com.ibm.ws.objectgrid.client.RemoteCacheLoader.sendCommandToPartition(RemoteCacheLoader.java:1232)
    at com.ibm.ws.objectgrid.client.RemoteCacheLoader.access$100(RemoteCacheLoader.java:150)
    at com.ibm.ws.objectgrid.client.RemoteCacheLoader$SendCommandToPartitionExecutor.run(RemoteCacheLoader.java:1013)
    at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1551)
    Caused by: org.omg.CORBA.INTERNAL: An internal error happened processing the request vmcid: 0x0 minor code: 0 completed: No
    at com.ibm.ws.objectgrid.ShardImpl.processMessage(ShardImpl.java:1081)
    at com.ibm.ws.objectgrid.partition.IDLShardPOA._invoke(IDLShardPOA.java:191)
    at com.ibm.CORBA.poa.POAServerDelegate.dispatchToServant(POAServerDelegate.java:435)
    at com.ibm.CORBA.poa.POAServerDelegate.internalDispatch(POAServerDelegate.java:362)
    at com.ibm.CORBA.poa.POAServerDelegate.dispatch(POAServerDelegate.java:258)
    at com.ibm.rmi.iiop.ORB.process(ORB.java:517)
    at com.ibm.CORBA.iiop.ORB.process(ORB.java:1575)
    at com.ibm.rmi.iiop.Connection.doRequestWork(Connection.java:2965)
    at com.ibm.rmi.iiop.Connection.doWork(Connection.java:2848)
    at com.ibm.rmi.iiop.WorkUnitImpl.doWork(WorkUnitImpl.java:64)
    at com.ibm.ws.objectgrid.thread.ORBThreadPool$1.run(ORBThreadPool.java:82)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
    at java.lang.Thread.run(Thread.java:769)
    Caused by: java.lang.ClassNotFoundException: com.ngc.grid.io.FileInputAgent
    at java.lang.Class.forName(Class.java:134)
    at com.ibm.ws.xs.io.ObjectStreamPool$ClassForNamePrivilegedAction.run(ObjectStreamPool.java:372)
    at com.ibm.ws.xs.io.ObjectStreamPool$ReusableInputStream.resolveClass(ObjectStreamPool.java:286)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1592)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1513)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1749)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1346)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:365)
    at com.ibm.ws.objectgrid.datagrid.BaseAgentCommand.inflateAgent(BaseAgentCommand.java:272)
    at com.ibm.ws.objectgrid.datagrid.BaseAgentCommand.readExternal(BaseAgentCommand.java:307)
    at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1809)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1768)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1346)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:365)
    at com.ibm.ws.objectgrid.util.SerializationHelper.readNullableObject(SerializationHelper.java:79)
    at com.ibm.ws.objectgrid.DistributedCommandImpl.readExternal(DistributedCommandImpl.java:181)
    at com.ibm.ws.objectgrid.DistributedCommandImpl.<init>(DistributedCommandImpl.java:113)
    at com.ibm.ws.objectgrid.event.DistributedCommandRequestEvent.readEvent(DistributedCommandRequestEvent.java:108)
    at com.ibm.ws.objectgrid.ServerCoreEventProcessor.initRequestEvent(ServerCoreEventProcessor.java:1924)
    at com.ibm.ws.objectgrid.ServerCoreEventProcessor.processClientServerRequest(ServerCoreEventProcessor.java:2041)
    at com.ibm.ws.objectgrid.ShardImpl.processMessage(ShardImpl.java:1033)
    ... 13 more

    I'm also attaching one of the ffdc files that is created when executed. All help, hints, and/or suggestions on this frustrating matter is appreciated.

    Thanks
    • lisaw
      lisaw
      101 Posts
      ACCEPTED ANSWER

      Re: Starting large amounts of containers with OSGi

      ‏2012-08-07T22:57:18Z  in response to wiedela
      For the first issue, the stack trace java.lang.ClassNotFoundException: zero.core.context.GlobalContext is because you have trace turned on. You don't have that component installed in the OSGi environment. It is use for problem determination, so that is why it is printing out for you. It does not cause any issue for concern. You can just turn off tracing if you don't want to see it.

      I know Andy answered this for you in another post, but for anyone else looking, check out this forum post for agents.

      • Lisa

      Websphere eXtreme Scale Development