Build a scalable, resilient, high performance database alternative with the ObjectGrid component of WebSphere Extended Deployment

This article will help you understand the basic concepts of the ObjectGrid component of IBM® WebSphere® Extended Deployment V6.1, describe how to build a scalable, resilient sample configuration using ObjectGrid as an in-memory distributed database -- one of several possible usage patterns -- and prepare you to begin architecting solutions that leverage ObjectGrid capabilities. This content is part of the IBM WebSphere Developer Technical Journal.

Share:

Alan Chambers, Consulting IT Specialist, IBM

Alan Chambers is a Consulting IT Specialist, in IBM's UK software business, specializing in advising customers in the Financial Services sector on WebSphere technologies, particularly WebSphere Application Server and WebSphere Extended Deployment. During Alan's time with IBM, he has repeatedly focused on new programming technologies, having been an early evangelist for Java in the mid-1990s and for Object Orientation and GUI development techniques before that. For some years, he was well known as an expert on the OS/2 operating system and its APIs, Presentation Manager, and Workplace Shell subsystems, and was the developer of the OS/2 Restricted Workplace Shell program offering.



07 November 2007

Introduction

Do you:

  • Have applications that require very high speed access to large quantities of data, for which traditional databases simply aren't fast enough?
  • Need to cache data from a database or other backend data store that needs to be accessed at speeds that would otherwise be unattainable?
  • Have applications that can be broken up into multiple parts, executing in parallel on different data, for which a grid solution would be ideal?
  • Have a requirement for application data to be propagated rapidly between geographically remote locations faster than traditional database replication?

If the answer to any of these is "yes," then ObjectGrid might be the answer.

This article introduces the ObjectGrid component of IBM WebSphere Extended Deployment V6.1, outlines its capabilities, and describes in detail how to set up a simple ObjectGrid configuration and connect it to a test application. You will also see how you can prove the scalability and failover features that ObjectGrid provides.

The examples here are written to run on a standalone Windows® system but can easily be adapted to run on any supported platform. The article assumes only a working knowledge of Java™ and the Windows operating system. To work with the sample programs, familiarity with Java development in Eclipse is helpful, although the samples can be run without it.


What is ObjectGrid?

ObjectGrid is a distributed in-memory data store for Java objects, designed for scalability, resilience, and high-performance. ObjectGrid has a variety of application uses, such as:

  • A high performance alternative to a traditional database, providing shared access for multiple threads or processes to a single consistent copy of data that is held only in memory.
  • A high performance cache for data persisted in another system (traditional database, enterprise application, and so on).
  • True grid processing, where an application is distributed across multiple processes such that execution is always co-located with the data, providing in-process data access performance.

The latest release of ObjectGrid has strengthened its effectiveness as an alternative to a traditional database in several areas, while also providing simplified configuration options.

ObjectGrid, a component of WebSphere Extended Deployment V6.1 (hereafter referred to as Extended Deployment), and can be purchased either as part of the full Extended Deployment product or as the component called WebSphere Extended Deployment Data Grid. It can be installed as an integral part of WebSphere Application Server or WebSphere Extended Deployment, used standalone without an application server, or run with any other Java application server.


How ObjectGrid works in a client server mode

The purpose of this article is to create an example of the first of the possible uses listed above: implement a distributed in-memory database, whose data is spread across multiple server instances and accessible from multiple client processes.

In previous releases of ObjectGrid, this was achieved using a static configuration, in which details of the location of the data and replicas are specified in configuration files. However, Version 6.1 introduced an alternative dynamic configuration, where decisions about data and replica placement are left to the ObjectGrid itself, and can change dynamically over time, as server instances are added or removed from the topology. These decisions can be left entirely to ObjectGrid or rules can be defined about placement; for example, to ensure that at least one replica of any data is located at a geographically different location, for disaster recovery purposes.

For dynamic configuration, an additional ObjectGrid component, the Catalog Server, makes placement and other decisions for the whole topology. (This example in this article uses a dynamic configuration.)

Where is the data and how is it accessed?

The simplest way to access and store data with ObjectGrid is by using an ObjectMap, which is essentially a Java map. Each item of data is associated with a key and both the key and the data can be any Java objects. For example, to insert a new item called "myData" in a map called "myMap" using the Java object "myKey" as the key, you code:

myMap.put(myKey, myData);

And to retrieve it, you would code:

Object myResultData = myMap.get(myKey);

Data is stored on the servers in a BackingMap and, by default, each client also has a local BackingMap, containing any data it has accessed. However, the application actually accesses an ObjectMap, which is created automatically when a transaction starts. This ObjectMap exists only for the life of the transaction; when the transaction is committed, any updates are written to the local BackingMap -- and also to the BackingMap on the servers, where the updated data is then accessible to other clients. This is illustrated in Figure 1.

Figure 1. Two applications accessing data in an ObjectGrid
Figure 1. Two applications accessing data in an ObjectGrid

In this example:

  • Two applications (App1 and App2) are accessing data in a Backing Map called "Account Bal," which contains account balances and uses the account number (a four-digit numeric string) as its key.
  • Each application has a local BackingMap, into which data is copied from the BackingMap on the server as it is accessed. In this illustration, the first application (App 1) has accessed account balances for accounts 0007, 0012, and 0015, and the second application (App 2) has accessed accounts 0004, 0007, and 0015.
  • The second application (App 2) is updating the account balance for account 0015, but has not yet committed the transaction. The new account balance is temporarily held in an ObjectMap. When the application commits the transaction, the new value will be copied to the local BackingMap and also to the BackingMap on the server. The ObjectMap will then be deleted.

Concurrency settings, comparable to those offered on traditional databases, ensure the right balance between consistency and performance for different styles of applications by the use of copy modes and locking strategies. These are described fully in the ObjectGrid documentation (see Resources).

How ObjectGrid achieves scalability and resilience

Scalability in an ObjectGrid is achieved by means of data partitioning and resilience is achieved by the use of replication.

In a typical configuration, ObjectGrid uses a hashcode derived from the key as the basis for dividing the data into partitions, and spreads the partitions across the server instances currently running. For example, a BackingMap containing customer data might be partitioned on the basis of customer surname; if you specify that you require 13 partitions, ObjectGrid might allocate names beginning A and B to one partition, C and D to the next and so on. If you run three ObjectGrid server processes, ObjectGrid it might place four partitions on two of them and five on the other, making 13 in total, as shown in Figure 2.

Figure 2. Thirteen partitions on three servers
Figure 2. Thirteen partitions on three servers

This behavior is dynamic, so that if a new server is started, some of the existing partitions or their replicas might relocate onto the new server, such that partitions and replicas are once again spread evenly across all the available servers, as shown in the example in Figure 3.

Figure 3. Thirteen partitions on four servers
Figure 3. Thirteen partitions on four servers

Examples vs. reality
In practice, ObjectGrid's algorithms are more sophisticated than this. ObjectGrid considers the placement of primary partitions and replicas, and avoids moving partitions when it is not necessary. In Figure 3, depending on the configuration of replicas, it is likely that additional replicas (not shown) would be created on the new server, rather than moving primaries as illustrated. Also, as the partitioning is, by default, based on hashcodes derived from the keys, it might not relate in such a simple way to the data as it does in these illustrations.

All you need to specify in the configuration files is the number of partitions, and ObjectGrid does the rest. In deciding how many partitions and how many servers might be required, you should consider such factors as:

  • Volume of data: The heap in each JVM sets a finite limit to the amount each server instance can hold, so the more data there is, the more servers you need to hold it all.
  • Resilience: By spreading the data across several servers, you can reduce the impact of any one server failing.
  • Scalability: You need to have enough partitions to make use of the number of servers you might be running initially and in the future. Servers can be added to a running cluster, but a change in the number of partitions requires a restart of all the servers, so it is good practice to set the number of partitions high enough to avoid restarting your topology.

Scalability

The ability to add servers to a running ObjectGrid cluster provides a very flexible mechanism for scalability. As load or data volumes increase, more servers can be added and the ObjectGrid will automatically re-balance across them all. Similarly, if a server has to be taken offline for any reason, or fails unexpectedly, the data is rebalanced onto the remaining servers. Clients communicate directly with the servers on which the data is located, so there is no need for a routing component that could itself become a bottleneck and limit scalability.

Since each server can hold up to 2GB of data on a 32 bit JVM (or much more on a 64 bit JVM), and you can have hundreds of servers running in a single ObjectGrid cluster, vast quantities of data can be held in memory.

Replication and failover

To protect against the failure of a process, and the loss of the data contained in any partitions located there, replicas are maintained for the partitions. A replica partition is automatically allocated on another server process, on another machine, if possible. As updates are made to the data in a partition, the replica is updated as well. If the original partition (the primary partition) fails, one of its replicas is promoted to become the new primary, and then a new replica is created somewhere else. All this happens in real time and the only effect on client applications is that some transactions might be rolled back while this is going on. You can choose how resilient your ObjectGrid will be by specifying an appropriate number of replicas on a large enough number of machines, and making them geographically dispersed (if disaster recovery is also a requirement).


Configuring the servers

Let's step through the installation and configuration of a simple ObjectGrid cluster consisting of five ObjectGrid servers, which will contain application data, and one ObjectGrid Catalog Server, a new component that manages the placement of data in a dynamic configuration. You will also write and run two small client applications to access data in the ObjectGrid and show how this configuration can recover from the failure of an ObjectGrid server.

About the example
The example described here can be used as a tutorial to learn the basics of ObjectGrid, or as the basis of a demonstration of its resilience and scalability. For clarity and simplicity, the sample used here is deliberately trivial and not typical of real applications. You could think of this as the ObjectGrid equivalent of a "Hello World" program.

Although other JVMs can be used, this article will configure ObjectGrid to run standalone with an IBM Java 5 runtime.

  1. Install the ObjectGrid runtime files

    If you have a full copy of WebSphere Extended Deployment V6.1 or WebSphere Extended Deployment Data Grid V6.1, install the Data Grid component into a convenient place in your file system, and locate the ObjectGrid directory (Figure 4). You can copy the ObjectGrid directory somewhere else if you wish or work with it where it is; its contents are not dependent in any way on the rest of the Extended Deployment installation.

    If you don't have the product version, you can download an evaluation copy of ObjectGrid. This version is fully functional for 60 minutes at a time -- after that time you have to restart all your ObjectGrid servers to get another 60 minutes, and so on, for as long as you like.

    Figure 4. Locate the ObjectGrid runtime files
    Figure 4. Locate the ObjectGrid runtime files

    Whichever way you obtained your ObjectGrid files, place the contents of the ObjectGrid directory somewhere convenient in your file system. For the purpose of this article, all text and figures refer to the ObjectGrid directory in c:\og61.

  2. Download and install the samples

    At this point you have two options:

    • You can proceed directly to the next steps, which explain how to set up a test ObjectGrid, and take you through creating the required configuration files and batch files, explaining each in turn and describing two sample applications to store and retrieve data from your ObjectGrid.
    • Alternatively, you can download the samples package included with this article, then continue on to the next sections for an understanding of the various files, without actually having to create them from scratch. If take this approach, download the dwsample.zip file and unpack it into the same directory as your ObjectGrid root directory. Having done this, the directory structure should be similar to that shown in Figure 5.
    Figure 5. Location of the sample files
    Figure 5. Location of the sample files
  3. Set the environment variables

    You must set several environment variables for ObjectGrid. Some of these can be set by means of the setupCmdLine batch file provided in the ObjectGrid\bin directory (do not modify this file), but you must also set JAVA_HOME and the PATH. For this example, we're running our samples from a directory under c:\og61 called "dwsample," and so a batch file called setup.bat was saved in the c:\og61\dwsample directory to set up the environment. If you haven't already done so, create the dwsample directory under c:\og61 and then create the setup.bat file inside it.

    This example uses the Java runtime provided with WebSphere Application Server V6.1, which is installed at c:\WAS61, but you can use any supported Java runtime; you will need to modify setup.bat accordingly. Listing 1 shows the setup.bat file for this environment. Remember that you will need to edit this file, taking into account the locations of your Java and ObjectGrid runtime files.

    Listing 1. Sample batch file to set the environment variables
    @rem
    @rem Set the JAVA_HOME environment variable to the Java runtime ObjectGrid will use.
    @rem
    
    set JAVA_HOME=c:\WAS61\java\jre
    
    @rem
    @rem Add the ObjectGrid and Java bin directories to the PATH
    @rem
    
    path=c:\og61\bin;%JAVA_HOME%\bin;%PATH%
    
    
    @rem
    @rem Run SetupCmdLine (in the ObjectGrid bin directory) to set
    @rem some more environment variables required by ObjectGrid
    @rem
    
    call setupCmdLine.bat
  4. Create the XML configuration files

    To start ObjectGrid servers, a batch file is provided called startOgServer.bat. For a dynamic configuration, you provide three parameters:

    1. A name you provide for this particular server instance.
    2. An XML file specifying some details of the ObjectGrids to be created, the backing maps, and so on (the ObjectGrid XML file).
    3. An XML file specifying partitioning and replication policies (the partitioning and deployment policies XML file) to be applied to the backing maps defined in the ObjectGrid XML file. (If the deployment policy XML file is not specified, one partition with no replication is assumed.)

    For a basic example, the two XML files can be quite simple. Create one ObjectGrid in which there will be only one BackingMap. Specify some settings for partitioning and replication, but otherwise accept defaults for everything else. Given this, an ObjectGrid XML file such as that shown in Listing 2 can be used. This is called "MyGrid.xml." This file simply names your ObjectGrid "MyObjectGrid," and your BackingMap as "MyBackingMap01."

    Listing 2. MyGrid.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <objectGridConfig xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://ibm.com/ws/objectgrid/config ../objectGrid.xsd"
            xmlns="http://ibm.com/ws/objectgrid/config">	
                <objectGrids>		
                    <objectGrid name="MyObjectGrid">			
                        <backingMap name="MyBackingMap01" />		
                    </objectGrid>	
                </objectGrids>
    </objectGridConfig>

    The second XML file, which provides partitioning and other deployment policy details, is shown in Listing 3, and is called "MyGridReplication.xml."

    Listing 3. MyGridReplication.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <deploymentPolicy xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation=
    	"http://ibm.com/ws/objectgrid/deploymentPolicy ../deploymentPolicy.xsd"
        xmlns="http://ibm.com/ws/objectgrid/deploymentPolicy">
    
        <objectgridDeployment objectgridName="MyObjectGrid">
            <mapSet name="MyMapSet" numberOfPartitions="20"
                minSyncReplicas="2" maxSyncReplicas="5"
                maxAsyncReplicas="0" numInitialContainers="3">
                <map ref="MyBackingMap01" />
            </mapSet>
        </objectgridDeployment>
    
    </deploymentPolicy>

    Here, you can see the ObjectGrid deployment details specified for the ObjectGrid called "MyObjectGrid" referred to on the ObjectGrid XML file above, and the reference to "MyBackingMap01," also as named in the ObjectGrid XML file. However, the deployment settings for this map are specified under the mapSet tag, which has been named "MyMapSet." A MapSet groups together a number of BackingMaps in an ObjectGrid that require the same partitioning and replication settings.

    Here, you can see the ObjectGrid deployment details specified for the ObjectGrid called MyObjectGrid referred to on the ObjectGrid XML file above, and the reference to MyBackingMap01, also as named in the ObjectGrid XML file. However, the deployment settings for this map are specified under the mapSet tag, which has been named MyMapSet. A MapSet groups together a number of BackingMaps in an ObjectGrid that require the same partitioning and replication settings.

    The MapSet settings are documented in detail in the ObjectGrid documentation but, in this example, we're specifying that the data is to be divided into 20 partitions, each of which will have a minimum of two synchronous replicas. This means that if a primary partition fails, there will always be an up to date replica that can take over, and, when that happens, another replica will already exist, so that there will never -- even for a brief interval -- be a situation where there is only one copy of that partition's data in existence. Since synchronous replicas have to be updated at the same time as the primary, and can therefore affect overall performance, it is important not to have too many, so we have specified a maximum of 5.

    Using asynchronous replicas is an alternative that avoids some of the performance overhead of synchronous replicas, but they are not guaranteed to be identical to the primary at all times. Asynchronous replicas are not used in this example, and so the maxAsynchReplicas value is set to zero.

    In this example, the number of initial containers is set to 3. This means that as ObjectGrid instances are started up, you don't want the allocation of partitions and replicas to start until at least three ObjectGrid servers are running. This avoids excessive workload on the catalog server in allocating (and then moving) partitions and replicas as ObjectGrid servers become available during start up. The default value of this setting is one more than the minimum number of sync replicas.

    If you have not already done so, create the two XML configuration files, MyGrid.xml and MyGridReplication.xml, as shown above, and place them in a new subdirectory called c:\og61\dwsample\config.

  5. Create batch files to start the ObjectGrid

    You are almost ready to start your ObjectGrid server instances, using the startOgServer.bat command. However, as this requires a fairly long command line, it is easier to create a batch file to start each of your five servers. For example, to start the first server instance, which is named "ogServ1," you can create a batch file similar to startsrv1.bat, shown in Listing 4 . (Enter this command as one line in the batch file; it is shown here split into multiple lines for display purposes.)

    Listing 4. startsrv1.bat
    startOgServer.bat ogServ1 -objectgridFile C:\og61\dwsample\config\MyGrid.xml 
    -deploymentPolicyFile C:\og61\dwsample\config\MyGridReplication.xml 
    -catalogServiceEndpoints localhost:2809

    Similarly, create batch files to start ogServ2, ogServ3, ogServ4, and ogServ5, and place all these batch files in the c:\og61\dwsample directory.

    In a dynamic configuration such as this, the ObjectGrid servers will not start unless a Catalog Server is already running. This is also started with the startOgServer.bat command, and for convenience, it is also probably worth creating a batch file containing this command. This file, startcat.bat in the c:\og61\dwsample directory, is shown in Listing 5, and starts a single instance of the Catalog Server, called cat1.

    Listing 5. startcat.bat
    startOgServer.bat cat1 -catalogServiceEndpoints cat1:localhost:6601:6602
  6. Start the ObjectGrid

    Using the dynamic configuration, as we are, you must start at least one Catalog Server before starting any ObjectGrid servers; if you attempt to start the latter first, it will fail with some CORBA errors, as it attempts to connect to a Catalog Server. Therefore:

    1. Start a command prompt window, change to the directory where your test files are located (c:\og61\dwsample) and run your setup.bat command to set the environment variables:

      setup

    2. Assuming that runs without errors, you can then run your startcat.bat command to start the catalog server:

      startcat

      This will take a few moments, during which information messages are displayed in the command prompt window. When it has finished starting, the last few lines of text will look similar to those shown in Figure 6.

      Figure 6. Messages indicating a successful Catalog Server start
      Figure 6. Messages indicating a successful Catalog Server start
    3. Next, run each of your startsrv?.bat commands to start five ObjectGrid servers:

      startsrv1
      startsrv2

      and so on.

      If you are running a personal firewall, watch out for any warnings, as these servers start to ensure that the firewall doesn't block communications between them and the Catalog Server. As each component starts, messages will be displayed in the command prompt window, and when it has started, it will look similar to Figure 7.

      Figure 7. Messages indicating a successful ObjectGrid Server start
      Figure 7. Messages indicating a successful ObjectGrid Server start

    You now have a resilient and scalable ObjectGrid running. The only weak point here is the Catalog Server, of which you have only started one instance. In a more realistic example, you would start at least two of these on separate machines, providing automatic failover, and thereby removing this remaining single point of failure. You would also, of course, run your ObjectGrid server instances on multiple machines.

Logs and traces

As each component started, you saw some messages appear on the command prompt window. The format of these is very similar to those produced by WebSphere Application Server and, similarly, each server instance also creates SystemOut.log and SystemErr.log files. These are found under the logs directory in subdirectories with names that match those of the servers concerned. There is also a ffdc directory, with first failure data capture data, and a Catalog Cluster directory, in which the Catalog Server logs are located. If tracing is enabled (via an option on the StartOgServer.bat command), then trace files are produced as well, again similar in style to those created by WebSphere Application Server. For this exercise, you will not enable tracing.

Figure 8 shows the directory structure after you have started your catalog and various ObjectGrid server instances.

Figure 8. Location of ObjectGrid log and trace files
Figure 8. Location of ObjectGrid log and trace files

Developing an ObjectGrid client application

You now have a distributed ObjectGrid, containing a map called MyBackingMap01, but containing no data. In some other scenarios, you might have developed a loader using the provided plug-in interface and referenced it in your ObjectGrid XML file. Loaders are typically used where ObjectGrid is being used as a cache for data held in some backend store, such as a database, and are responsible for loading data from and propagating updates to the backend store. (This is the second usage pattern referred to earlier.)

However, in the spirit of simplicity, assume that the data is to be created by one application and read by another; in other words, there is no external source of the data. The applications will be kept extremely simple, just inserting some trivial data under a range of keys into your map, and then retrieving it.

Connecting to the ObjectGrid and your map

In order to access your backing map, you have to locate it with the help of the ObjectGrid Catalog Server. The code shown in Listing 6 connects to the Catalog Server (using its default port 2809, on the local machine), and uses the context obtained to obtain a reference to the ObjectGrid (MyObjectGrid, as it was named in the ObjectGrid XML file). Having obtained a session for the ObjectGrid, the code uses the session to gain an ObjectMap reference to the Backing Map you rdefined (MyBackingMap01).

Listing 6. Code to access an ObjectMap
ObjectGridManager ogMgr = ObjectGridManagerFactory.getObjectGridManager();
ClientClusterContext myCtx = ogMgr.connect("localhost:2809", null, null);
ObjectGrid myOG = ogMgr.getObjectGrid(myCtx, "MyObjectGrid");
Session sess = myOG.getSession();
ObjectMap myMap = sess.getMap("MyBackingMap01");

Putting data into your ObjectGrid

You can now insert some data into the map, as shown in Listing 7. The code inserts 200 values, using Integer keys numbered 0 to 199. The associated data is simple: a string "Hello World " appended with the key number.

Listing 7. Code to insert data into the BackingMap
int i;
for (i = 0; i < 200; i++) {
	sess.begin();
	Integer key = new Integer(i);
	Object o = myMap.get(key);
	if (o == null) {
		myMap.put(new Integer(i), "Hello World " + i);
	}
	sess.commit();
}

Notice that before each insertion, a new transaction is started by invoking the begin() method on the Session object you obtained above and, after putting the data, you invoke the commit() method. The commit() causes the data concerned to be copied from the ObjectMap associated with the transaction into the application's local BackingMap, and also to the BackingMap on the servers, where it becomes visible to other applications.

It is not always necessary to code the begin() and commit() method calls on the Session object, as ObjectGrid provides autocommit support for methods called on an ObjectMap outside the scope of a Session transaction. For these examples, the transactions are defined explicitly to illustrate the use of these methods, which would usually be needed in more realistic applications scenarios.

This trivial example checks whether an entry already exists with the current key before writing new data, in order to avoid an exception that would occur if data with this key already existed.

Be aware that several of the calls here can generate Java exceptions and would therefore normally be wrapped with a try/catch clause. This is omitted here in the interest of clarity; instead, the whole sample is contained in a method that throws a generic exception, which avoids the compile time errors that would otherwise be generated. See the complete program in Listing 8.

Listing 8. The complete application to put data into the ObjectGrid (MyOgClientWriter.java)
package ogdemo;

import com.ibm.websphere.objectgrid.ClientClusterContext;
import com.ibm.websphere.objectgrid.ObjectGrid;
import com.ibm.websphere.objectgrid.ObjectGridManager;
import com.ibm.websphere.objectgrid.ObjectGridManagerFactory;
import com.ibm.websphere.objectgrid.ObjectMap;
import com.ibm.websphere.objectgrid.Session;

public class MyOgClientWriter {

	public static void main(String[] args) {

		try {
			ogtest();
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

	private static void ogtest() throws Exception {
		ObjectGridManager ogMgr = 
                          ObjectGridManagerFactory.getObjectGridManager();

		ClientClusterContext myCtx = 
                          ogMgr.connect("localhost:2809", null, null);
		ObjectGrid myOG = ogMgr.getObjectGrid(myCtx, "MyObjectGrid");
		Session sess = myOG.getSession();
		ObjectMap myMap = sess.getMap("MyBackingMap01");

		int i;
		for (i = 0; i < 200; i++) {
			sess.begin();
			Integer key = new Integer(i);
			Object o = myMap.get(key);
			if (o == null) {
				myMap.put(new Integer(i), "Hello World " + i);
			}
			sess.commit();
		}

		System.out
                  .println("\n\n*** Finished loading data into ObjectMap ***");

	}
}

Accessing data from the ObjectGrid

Accessing data in the backing map within your ObjectGrid is very similar to the code above for inserting data. The steps to access the ObjectGrid, and the Session and BackingMap objects are the same. Retrieving data is illustrated in Listing 9.

Listing 9. Code to access data from the ObjectGrid, showing partition numbers
BackingMap bm = myOG.getMap("MyBackingMap01");
PartitionManager pm = bm.getPartitionManager();
	
for (i = 0; i < 200; i++) {
	Integer key = new Integer(i);
	Object o = myMap.get(key);
	if (o != null) {
		System.out.println("Key " + i 
                                       + "   value: " + o.toString() 
                                       + "  Partition number: " 
                                       + pm.getPartition(key));
	}
}

Just to make it more interesting, we have included an indication of the partition number each particular item of data was assigned to. The example also shows autocommit being used instead of an explicit Session.begin() and Session.commit() method call.

Listing 10 shows the complete client application (MyOgClientReader.java).

Listing 10. The application to access data from the ObjectGrid (MyOgClientReader.java)
package ogdemo;
import com.ibm.websphere.objectgrid.BackingMap;
import com.ibm.websphere.objectgrid.ClientClusterContext;
import com.ibm.websphere.objectgrid.ObjectGrid;
import com.ibm.websphere.objectgrid.ObjectGridManager;
import com.ibm.websphere.objectgrid.ObjectGridManagerFactory;
import com.ibm.websphere.objectgrid.ObjectMap;
import com.ibm.websphere.objectgrid.PartitionManager;
import com.ibm.websphere.objectgrid.Session;

public class MyOgClientReader {

	public static void main(String[] args) {

		try {
			ogtest();
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

	private static void ogtest() throws Exception {
		ObjectGridManager ogMgr =ObjectGridManagerFactory.getObjectGridManager();
		ClientClusterContext myCtx = ogMgr.connect("localhost:2809", null, null);
		ObjectGrid myOG = ogMgr.getObjectGrid(myCtx, "MyObjectGrid");
		Session sess = myOG.getSession();
		ObjectMap myMap = sess.getMap("MyBackingMap01");

		BackingMap bm = myOG.getMap("MyBackingMap01");
		PartitionManager pm = bm.getPartitionManager();

		int i;
		for (i = 0; i < 200; i++) {

			Integer key = new Integer(i);
			Object o = myMap.get(key);
			if (o != null) {
				System.out.println("Key " 
                                  + i 
                                  + "   value: " 
                                  + o.toString()
                                  + "  Partition number: " 
                                  + pm.getPartition(key));
			}
		}
	}
}

Running the client applications

  • From a command prompt

    If you simply want to run the samples provided with this article and experiment with failover (described later), you can do so by running them from a command prompt, using the batch files provided. To load some data into the ObjectGrid, run the following batch file from the command prompt:

    writesample.bat

    It takes a few seconds to run, after which, you will see the message "Finished loading data into ObjectMap" (Listing 11).

    Listing 11. Output from the MyClientWriter application
    C:\og61\dwsample>c:\was61\java\bin\java –classpath 
    myclient.jar;C:\og61\lib\ogclient.jar ogdemo.MyOgClientWriter
    [10/07/07 16:34:30:828 BST]  1420142 RuntimeInfo   I CWOBJ2604I: The stream query jar file
    is not in the class path.
    [10/07/07 16:34:35:093 BST]  1420142 ObjectGridImp I CWOBJ1308I: Security of the 
    ObjectGrid instance MyObjectGrid is disabled.
    
    *** Finished loading data into ObjectMap ***

    To retrieve data from the ObjectGrid, run the following batch file from the command prompt:

    readsample.bat

    This also takes a few moments, after which you should see a list of the 200 entries in the BackingMap that were created by the previous program, along with the partition number associated with each entry. The first few lines of the output are illustrated in Listing 12.

    Listing 12. Some output from the MyOgClientReader application
    C:\og61\dwsample>c:\was61\java\bin\java -classpath 
    myclient.jar;C:\og61\lib\ogclient.jar ogdemo.MyOgClientReader
    [10/07/07 16:39:57:640 BST]  1420142 RuntimeInfo   I CWOBJ2604I: The stream query jar file
    is not in the class path.
    [10/07/07 16:40:01:734 BST]  1420142 ObjectGridImp I CWOBJ1308I: Security of the 
    ObjectGrid instance MyObjectGrid is disabled.
    Key 0   value: Hello World 0  Partition number: 0
    Key 1   value: Hello World 1  Partition number: 1
    Key 2   value: Hello World 2  Partition number: 2
    Key 3   value: Hello World 3  Partition number: 3
    Key 4   value: Hello World 4  Partition number: 4
    Key 5   value: Hello World 5  Partition number: 5
    Key 6   value: Hello World 6  Partition number: 6
    Key 7   value: Hello World 7  Partition number: 7
    Key 8   value: Hello World 8  Partition number: 8
    Key 9   value: Hello World 9  Partition number: 9
    Key 10   value: Hello World 10  Partition number: 10
    Key 11   value: Hello World 11  Partition number: 11
    Key 12   value: Hello World 12  Partition number: 12
    Key 13   value: Hello World 13  Partition number: 13
    Key 14   value: Hello World 14  Partition number: 14
  • From Eclipse or Rational Application Developer

    If you are developing your client application using Eclipse or an Eclipse-based tool, such as IBM Rational® Application Developer, you need to ensure that the Java project has the file ogclient.jar in its build path and classpath. This file is found in the ObjectGrid lib directory; in this case, c:\og61\lib.

    To set the buildpath in Eclipse:

    1. Open the Properties panel for your Java project.
    2. Select Java Build Path from the list of properties.
    3. Use Add External JARs... to add a reference to the ogclient.jar file in the lib directory.

    After you do this step, the Properties dialog should look similar to Figure 9.

    Figure 9. Setting the Java build path in Eclipse
    Figure 9. Setting the Java build path in Eclipse

    If your Eclipse-based development tool is configured to run on a non-IBM Java runtime, you will have to implement some IBM-provided files in the JRE to avoid an incompatibility in the CORBA implementation. Details are provided in the ObjectGrid documentation. Still, you should specify an IBM runtime instead, if you have one, for a first test. If your Eclipse environment is running on another JRE, you can specify an IBM runtime for applications you launch. To do this, go to the Run... configuration dialog for your application, and select an IBM JRE (you might have to use the Installed JREs button to locate the IBM JRE in the file system).

    Figure 10. Setting the test configuration in Eclipse to use the IBM JRE
    Figure 10. Setting the test configuration in Eclipse to use the IBM JRE

Resilience: What happens when a process fails?

You can prove to yourself that partition failover works by trying the following:

  1. As you start the Catalog Server and each of the ObjectGrid servers, notice the process IDs assigned by the operating system to each. You can do this in Windows by watching the Task List window (on the Processes tab), as you start each component, and look for instances of java.exe. As each component starts, two Java processes start but one then disappears; the remaining one is the actual server process.

  2. Run a test client application to load some data into the ObjectGrid, using a range of keys so that data is placed in all your partitions. You can do this by running the writesample.bat batch file or by running the MyOgClientWriter application from within your development environment.

  3. Run a test client application to query and list the data. You can do this from the command prompt by running the readsample.bat batch file, or from your development tool by running the MyOgClientReader application.

  4. Using the Task Manager End Process button, kill one of the ObjectGrid server processes.

  5. Run the test client application again to query and list the data. Notice that none of the data is missing.

  6. If you remain unconvinced, restart the server instance you killed noting its process ID as before, then kill another one. Repeat this until you've killed and restarted them all one by one, proving to yourself that the data is intact at each step in the process.

Be careful not to kill the Catalog Server process -- or any Java processes related to your Eclipse environment. In a production environment, multiple Catalog Servers would be run to provide failover, but you are only running one instance in this example, so if you kill it, the application will no longer work.


Conclusion

This article described one of several patterns for using ObjectGrid: as an in-memory distributed database. A minimal set of configuration files were required to setup a simple test ObjectGrid consisting of five servers and two small Java applications to insert data into and extract data from the ObjectGrid. Finally, you learned how you can see ObjectGrid's failover capabilities in action.

ObjectGrid has many more advanced capabilities not described in this article, but hopefully this introduction provided you with a starting point for learning more about the capabilities of this advanced component.


Acknowledgements

The author thanks:

  • Jonathan Marshall, Senior IT Specialist, IBM UK Software Business, for his help in understanding ObjectGrid and for being a sounding board for ideas.
  • Matthew Perrins, Senior Consulting IT Specialist, IBM Hursley Lab Services, for suggesting this article and for support and encouragement.
  • Stuart Foster, Consulting IT Specialist, IBM UK Software Business, for his very thorough proof reading of this article and for road-testing the samples.
  • Billy Newport and Jared Anderson from ObjectGrid Development for the excellent support they have provided.

Download

DescriptionNameSize
Code sampledwsample.zip9 KB

Resources

Learn

Get products and technologies

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=266675
ArticleTitle=Build a scalable, resilient, high performance database alternative with the ObjectGrid component of WebSphere Extended Deployment
publish-date=11072007