Topic
  • 4 replies
  • Latest Post - ‏2012-12-05T18:48:49Z by SystemAdmin
SystemAdmin
SystemAdmin
1485 Posts

Pinned topic problems with 8.5.0.1 catalog server interaction.

‏2012-11-30T14:35:50Z |
Hi - I am trying to configure a minimal PoC for ObjectGrid, basic barebones to demonstrate the simple concepts.
When doing so, I am unable to use xsadmin utility to query the contents of the grid. My objectgridserver.properties file specifically sets enableQuorum=false but after starting the server, the output on stdout states that quorum is enabled:
INFO: LAUNCHING_CATALOG_SERVICE_CWOBJ2518I
11/30/12 8:26:31:861 CST 0000001a ServerImpl I CWOBJ2518I: Launching ObjectGrid catalog service: localhost.
Nov 30, 2012 8:26:31 AM com.ibm.ws.objectgrid.ServerImpl

INFO: QUORUM_ENABLED_CWOBJ1251I
11/30/12 8:26:31:866 CST 0000001a ServerImpl I CWOBJ1251I: Quorum is enabled for the catalog service.
When executing xsadmin -containers, get a reply that the catalog server is waiting on quorum. When override, get Java NPE.
C:\apps\IBM\WebSphere\eXtremeScale_8501\ObjectGrid\bin>xsadmin -containers

This administrative utility is provided as a sample only and is not to be
considered a fully supported component of the WebSphere eXtreme Scale product

Starting at: 2012-11-30 08:29:19.000000633
Connecting to Catalog service at localhost:1099
Command failed. Quorum is enabled and catalog server is waiting for quorum. Chec
k your environment to determine if it is undergoing a brown out or black out. If
you detect a brown out then try the command again at a later time. If you detec
t a black out consider overriding quorum.

C:\apps\IBM\WebSphere\eXtremeScale_8501\ObjectGrid\bin>xsadmin -overridequorum

This administrative utility is provided as a sample only and is not to be
considered a fully supported component of the WebSphere eXtreme Scale product

Starting at: 2012-11-30 08:29:33.000000469
Connecting to Catalog service at localhost:1099
Performed overrideQuorum call

The following Exception was caught during execution, javax.management.RuntimeMBe
anException: java.lang.NullPointerException

This error may be caused by incorrect host, port, or configuration arguments
- Please consult the help statement for more information
javax.management.RuntimeMBeanException: java.lang.NullPointerException
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(Default
MBeanServerInterceptor.java:851)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBe
anException(DefaultMBeanServerInterceptor.java:864)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultM
BeanServerInterceptor.java:833)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:803
)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnecti
onImpl.java:1458)
at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectio
nImpl.java:100)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run
(RMIConnectionImpl.java:1303)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(R
MIConnectionImpl.java:1391)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImp
l.java:823)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:339)
at sun.rmi.transport.Transport$1.run(Transport.java:189)
at sun.rmi.transport.Transport$1.run(Transport.java:186)
at java.security.AccessController.doPrivileged(AccessController.java:314
)
at sun.rmi.transport.Transport.serviceCall(Transport.java:185)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:5
65)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTranspor
t.java:820)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport
.java:679)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1121)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:614)
at java.lang.Thread.run(Thread.java:777)
at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Unknow
n Source)
at sun.rmi.transport.StreamRemoteCall.executeCall(Unknown Source)
at sun.rmi.server.UnicastRef.invoke(Unknown Source)
at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl_Stub.invoke(Unknown Sou
rce)
at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.
invoke(Unknown Source)
at com.ibm.websphere.samples.objectgrid.admin.JMXProxy.invoke(JMXProxy.j
ava:58)
at $Proxy1.overrideQuorum(Unknown Source)
at com.ibm.websphere.samples.objectgrid.admin.OGAdmin.main(OGAdmin.java:
1479)
Caused by: java.lang.NullPointerException
at com.ibm.ws.objectgrid.catalog.CatalogMessagePropagatorFactory$Catalog
MessagePropagator.sendMessage(CatalogMessagePropagatorFactory.java:122)
at com.ibm.ws.objectgrid.quorum.QuorumManager.overrideQuorum(QuorumManag
er.java:236)
at com.ibm.ws.objectgrid.catalog.placement.PlacementServiceImplMBean.inv
oke(PlacementServiceImplMBean.java:632)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultM
BeanServerInterceptor.java:831)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:803
)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnecti
onImpl.java:1458)
at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectio
nImpl.java:100)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run
(RMIConnectionImpl.java:1303)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(R
MIConnectionImpl.java:1391)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImp
l.java:823)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:339)
at sun.rmi.transport.Transport$1.run(Transport.java:189)
at sun.rmi.transport.Transport$1.run(Transport.java:186)
at java.security.AccessController.doPrivileged(AccessController.java:314
)
at sun.rmi.transport.Transport.serviceCall(Transport.java:185)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:5
65)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTranspor
t.java:820)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport
.java:679)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1121)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:614)
at java.lang.Thread.run(Thread.java:777)
Ending at: 2012-11-30 08:29:33.000000948

Lance
www lancelotlinc com
Updated on 2012-12-05T18:48:49Z at 2012-12-05T18:48:49Z by SystemAdmin
  • Art_Jolin
    Art_Jolin
    26 Posts

    Re: problems with 8.5.0.1 catalog server interaction.

    ‏2012-12-04T20:37:14Z  
    Please append the body of your objectgridserver.properties file (it should be fairly small, a few dozen lines, right?) and also the startOgServer command where you pass this props file (or do you pass it another way?). In short, I'm suggesting we verify you are indeed using this props file and not some other that does have quorum enabled, and that there is no typo in the props file.
  • SystemAdmin
    SystemAdmin
    1485 Posts

    Re: problems with 8.5.0.1 catalog server interaction.

    ‏2012-12-04T21:22:52Z  
    • Art_Jolin
    • ‏2012-12-04T20:37:14Z
    Please append the body of your objectgridserver.properties file (it should be fairly small, a few dozen lines, right?) and also the startOgServer command where you pass this props file (or do you pass it another way?). In short, I'm suggesting we verify you are indeed using this props file and not some other that does have quorum enabled, and that there is no typo in the props file.
    objectgridserver.properties:

    #------------------------------------------------------------------------------
    1. Enable or disable quorum for the catalog service. Quorum is used to ensure
    2. that a majority of the catalog service grid is available before allowing
    3. modification to the placement of partitions on available container servers.
    #
    1. Set enableQuorum to "true" or "enabled" will enable the catalog service quorum.
    2. By default, the quorum is disabled.
    #
    1. This property only applies to a WebSphere eXtreme Scale catalog service.
    #
    1. Uncomment this property to enable or disable quorum.
    #------------------------------------------------------------------------------
    enableQuorum=false

    Invocation (by java):

    System.out.println("Starting CacheInit service. "+ config.getServletContext().getContextPath() );
    ServerProperties props = ServerFactory.getServerProperties();
    props.setCatalogServiceBootstrap("localhost:1099");
    props.setServerName("localhost"); // name server
    props.setTraceSpecification("com.ibm.ws.objectgrid=all=enabled"); // TraceSpec

    CatalogServerProperties catalogProps =
    ServerFactory.getCatalogProperties();
    catalogProps.setCatalogServer(true); // enable catalog service
    catalogProps.setQuorum(true); // enable quorum

    Server server = (Server) ServerFactory.getInstance();

    DeploymentPolicy policy = DeploymentPolicyFactory.createDeploymentPolicy
    (new URL("file:///TEMP/deployment.xml"),
    new URL("file:///TEMP/objectgrid.xml"));
    Container container = server.createContainer(policy);
    From stdout:

    INFO: LOADED_PROPERTY_FILES_CWOBJ0913
    12/4/12 15:25:23:638 CST 0000001a ServerFactory I CWOBJ0913I: Server propert
    y files have been loaded: C:/TEMP/objectgridserver.properties.
  • jhanders
    jhanders
    261 Posts

    Re: problems with 8.5.0.1 catalog server interaction.

    ‏2012-12-04T22:21:23Z  
    objectgridserver.properties:

    #------------------------------------------------------------------------------
    1. Enable or disable quorum for the catalog service. Quorum is used to ensure
    2. that a majority of the catalog service grid is available before allowing
    3. modification to the placement of partitions on available container servers.
    #
    1. Set enableQuorum to "true" or "enabled" will enable the catalog service quorum.
    2. By default, the quorum is disabled.
    #
    1. This property only applies to a WebSphere eXtreme Scale catalog service.
    #
    1. Uncomment this property to enable or disable quorum.
    #------------------------------------------------------------------------------
    enableQuorum=false

    Invocation (by java):

    System.out.println("Starting CacheInit service. "+ config.getServletContext().getContextPath() );
    ServerProperties props = ServerFactory.getServerProperties();
    props.setCatalogServiceBootstrap("localhost:1099");
    props.setServerName("localhost"); // name server
    props.setTraceSpecification("com.ibm.ws.objectgrid=all=enabled"); // TraceSpec

    CatalogServerProperties catalogProps =
    ServerFactory.getCatalogProperties();
    catalogProps.setCatalogServer(true); // enable catalog service
    catalogProps.setQuorum(true); // enable quorum

    Server server = (Server) ServerFactory.getInstance();

    DeploymentPolicy policy = DeploymentPolicyFactory.createDeploymentPolicy
    (new URL("file:///TEMP/deployment.xml"),
    new URL("file:///TEMP/objectgrid.xml"));
    Container container = server.createContainer(policy);
    From stdout:

    INFO: LOADED_PROPERTY_FILES_CWOBJ0913
    12/4/12 15:25:23:638 CST 0000001a ServerFactory I CWOBJ0913I: Server propert
    y files have been loaded: C:/TEMP/objectgridserver.properties.
    Based off of your configuration and code the objectGridServer.properties has enableQuorum=false, but you override that programmatically and set it to true in your code catalogProps.setQuorum(true); // enable quorum. ServerProperties and CatalogServerProperties are pre-populated with the contents of objectGridServer.properties, but you can override what is set in the properties programatically. That is the precedence rules. The last operation was to set it to true so that is what we use when running.

    Also since you are using 8.5 function you should make use of the xscmd script instead of the xsadmin script. xscmd productizes the functions in xsadmin and provides a fully supported administrative utility instead of the xsadmin script that provided users with an example of how to do it themselves.

    I hope that helps.

    Jared Anderson
  • SystemAdmin
    SystemAdmin
    1485 Posts

    Re: problems with 8.5.0.1 catalog server interaction.

    ‏2012-12-05T18:48:49Z  
    • jhanders
    • ‏2012-12-04T22:21:23Z
    Based off of your configuration and code the objectGridServer.properties has enableQuorum=false, but you override that programmatically and set it to true in your code catalogProps.setQuorum(true); // enable quorum. ServerProperties and CatalogServerProperties are pre-populated with the contents of objectGridServer.properties, but you can override what is set in the properties programatically. That is the precedence rules. The last operation was to set it to true so that is what we use when running.

    Also since you are using 8.5 function you should make use of the xscmd script instead of the xsadmin script. xscmd productizes the functions in xsadmin and provides a fully supported administrative utility instead of the xsadmin script that provided users with an example of how to do it themselves.

    I hope that helps.

    Jared Anderson
    Thank you for your reply. Yes, that makes alot of sense.