Terracotta Clustering Configuration Elements

Edit online

The terracotta and terracottaConfig elements in the Ehcache configuration control the clustering of caches with Terracotta.

terracotta

Edit online

The <terracotta> element is an optional sub-element of <cache>. It can be set differently for each <cache> defined in ehcache.xml.

<terracotta> has one sub-element, <nonstop>. For more information about the <nonstop> element, see Configuring Nonstop Cache.

The following <terracotta> attribute allows you to control the type of data consistency for the distributed cache:

  • consistency – Uses no cache-level locks for better performance ("eventual" DEFAULT) or uses cache-level locks for immediate cache consistency ("strong"). When set to "eventual", allows reads without locks, which means the cache may temporarily return stale data in exchange for substantially improved performance. When set to "strong", guarantees that after any update is completed no local read can return a stale value, but at a potentially high cost to performance, because a large number of locks may need to be stored in client and server heaps.

    Once set, this consistency mode cannot be changed except by reconfiguring the cache using a configuration file and reloading the file. This setting cannot be changed programmatically. For more information, see Consistency Modes.

Except for special cases, the following <terracotta> attributes should operate with their default values:

  • clustered – Enables ("true" DEFAULT) or disables ("false") Terracotta clustering of a specific cache. Clustering is enabled if this attribute is not specified. Disabling clustering also disables the effects of all of the other attributes.
  • localCacheEnabled – Enables ("true" DEFAULT) or disables ("false") local caching of distributed cache data, forcing all of that cached data to reside on the Terracotta Server Array. Disabling local caching may improve performance for distributed caches in write-heavy use cases.
  • synchronousWrites – Enables ("true") or disables ("false" DEFAULT) synchronous writes from Terracotta clients (application servers) to Terracotta servers. Asynchronous writes (synchronousWrites="false") maximize performance by allowing clients to proceed without waiting for a "transaction received" acknowledgment from the server. This acknowledgment is unnecessary in most use cases. Synchronous writes (synchronousWrites="true") provide extreme data safety but at a performance cost by requiring that a client receive server acknowledgment of a transaction before that client can proceed. If the cache's consistency mode is eventual, or while it is set to bulk load using the bulk-load API, only asynchronous writes can occur (synchronousWrites="true" is ignored).
  • concurrency – Sets the number of segments for the map backing the underlying server store managed by the Terracotta Server Array. If concurrency is not explicitly set (or set to "0"), the system selects an optimized value.

The following attributes are used with Hibernate:

  • localKeyCache – Enables ("true") or disables ("false" DEFAULT) a local key cache. BigMemory Max can cache a "hotset" of keys on clients to add locality-of-reference, a feature suitable for read-only cases. Note that the set of keys must be small enough for available memory.
  • localKeyCacheSize – Defines the size of the local key cache in number (positive integer) of elements. In effect if localKeyCache is enabled. The default value, 300000, should be tuned to meet application requirements and environmental limitations.
  • orphanEviction – Enables ("true" DEFAULT) or disables ("false") orphan eviction. Orphans are cache elements that are not resident in any Hibernate second-level cache but still present on the cluster's Terracotta server instances.
  • orphanEvictionPeriod – The number of local eviction cycles (that occur on Hibernate) that must be completed before an orphan eviction can take place. The default number of cycles is 4. Raise this value for less aggressive orphan eviction that can reduce faulting on the Terracotta server, or lower it if garbage on the Terracotta server is a concern.

Default Behavior

By default, adding <terracotta/> to a <cache> block is equivalent to adding the following:

<cache name="sampleTerracottaCache" 
    maxEntriesLocalHeap="1000" 
    eternal="false" 
    timeToIdleSeconds="3600" 
    timeToLiveSeconds="1800""> 
  <persistence strategy="distributed"/> 
  <terracotta clustered="true" consistency="eventual" /> 
</cache>

terracottaConfig

Edit online

The <terracottaConfig> element enables the distributed-cache client to identify a source of Terracotta configuration. It also allows a client to rejoin a cluster after disconnecting from that cluster and being timed out by a Terracotta server. For more information on the rejoin feature, see the BigMemory Max High-Availability Guide. In addition, the <terracottaConfig> element allows you to enable caches for WAN replication.

The client must load the configuration from a file or a Terracotta server. The value of the url attribute should contain a path to the file, a system property, or the address and TSA port (9510 by default) of a server.

Note: In a Terracotta cluster, the application server is also known as the client.

For more information on client configuration, see the "Terracotta Configuration Parameters" in the BigMemory Max Administrator Guide.

Adding an URL Attribute

Add the url attribute to the <terracottaConfig> element as follows:

<terracottaConfig url="<source>" />

where <source> must be one of the following:

  • A path (for example, url="/path/to/tc-config.xml")
  • An URL (for example, url="http://www.mydomain.com/path/to/tc-config.xml)
  • A system property (for example, url="${terracotta.config.location}"), where the system property is defined like this:
System.setProperty("terracotta.config.location","10.x.x.x:9510"");
  • A Terracotta host address in the form <host>:<tsa-port> (for example, url="host1:9510"). Note the following about using server addresses in the form <host>:<tsa-port>:
    • The default TSA port is 9510.
    • In a multi-server cluster, you can specify a comma-delimited list (for example, url="host1:9510,host2:9510,host3:9510").
    • If the Terracotta configuration source changes at a later time, it must be updated in configuration.

Adding the WAN Attribute

For each cache to be replicated, its ehcache.xml configuration file must include the wanEnabledTSA attribute set to "true" within the <terracottaConfig> element. Add the wanEnabledTSA attribute to the <terracottaConfig> element as follows:

<terracottaConfig wanEnabledTSA="true"/>

For more information, see the WAN Replication User Guide.

Embedding Terracotta Configuration

You can embed the contents of a Terracotta configuration file in ehcache.xml as follows:

<terracottaConfig> 
   <tc-config> 
       <servers> 
           <server host="server1" name="s1"/> 
           <server host="server2" name="s2"/> 
       </servers> 
       <clients> 
           <logs>app/logs-%i</logs> 
       </clients> 
   </tc-config> 
</terracottaConfig>