See the WebSphere eXtreme Scale Wiki for links to eXtreme Scale Version 7.0 documentation.
If you log in
with your developerWorks ID, you can leave comments and feedback for the development team.
Use partitioning if the ObjectGrid needs to store more data than fits into the Java Virtual Machine (JVM) heap. Partitioning is accomplished by splitting the data using an application-specified scheme. The partitioning scheme is derived from the key.
A MapSet is a collection of maps that are typically used together. The maps are partitioned together using a common key. Each map in the map set must have a key from which a common key can be derived. The partitioning is done on this common key. A common key is necessary only when routing from a client.
Map APIs and partitioning
All maps in a MapSet must use keys that include a common partitioning key. With this common key, data in any of the maps can be found from a client.
Entities and partitioning
EntityManager entities have an optimization that helps clients that are working with entities on a server. The entity schema on the server for the MapSet can specify a single root entity. The client must access all entities through the root entity. The EntityManager can then find related entities from that root in the same partition without requiring the related maps to have a common key. The root entity establishes affinity with a single partition. This partition is used for all entity fetches within the transaction after affinity is established. This affinity can save memory because the related maps do not require a common key. The root entity must be specified with a modified entity annotation as in the following example.
This entity should normally be used to find the root of the object graph and all children are assumed to be in the same partition as the root. The children in this graph are only accessible from a client from the root entity.
Root entities are always required in partitioned environments when using an ObjectGrid client to communicate to the server. Only one root entity type can be defined per ObjectGrid.
Root entities are not required when using Extreme Transaction Processing (XTP) style ObjectGrids, since all communication to the ObjectGrid partition is accomplished through direct, local access and not the client/server mechanism.
How many partitions should an application use?
A grid can have many partitions, or thousands if required. A grid can scale up to the product of the number of partitions times the number of shards per partition. For example, if you have 16 partitions and each partition has one primary and one replica, or two shards, then you can potentially scale to 32 JVMs. In this case one shard is defined for each JVM. You must choose a reasonable number of partitions based on the expected number of server JVMs that are likely to be used. Each shard carries overhead for the system. The system is designed to scale out to handle this overhead in line with how many server JVMs are available.
Applications should not use thousands of partitions if the application runs on a grid of four container JVMs. The application should be configured to have a reasonable number of shards for each container JVM. For example, an unreasonable configuration is 2000 partitions with two shards that are running on four container JVMs. This configuration would result in 4000 shards placed on four container JVMs or 1000 shards per container JVM.
A better configuration would be under 10 shards for each expected container JVM. This configuration still gives the possibility of scaling ten times the initial configuration while keeping a reasonable number of shards per container JVM.
Consider this scaling example: you currently have six boxes with two container JVMs per box. You expect to grow to 20 boxes over the next three years. With 20 boxes, you have 40 container JVMs, and choose 60 to be pessimistic. You want 4 shards per container JVM. You have 60 potential containers, or a total of 240 shards. If you have a primary and replica per partition, then you want 120 partitions. This example gives you 240 divided by 12 container JVMs, or 20 shards per container JVM for the initial deployment with the potential to scale out to 20 boxes later.
© Copyright IBM Corporation 2007,2009. All Rights Reserved.
