This blog promotes knowledge sharing through experience and collaboration. For more product information, visit our WebSphere Commerce CSE page. For easier navigation, utilize the Categories to find posts that match your interest.
Cache Invalidation Using Kafka Zookeeper Messaging System In V9
In this post we will look at Cache Invalidation when using Commerce Remote Store (CRS, Store Server). Traditionally in WebSphere Commerce versions 7 and 8, the DynaCacheInvalidation command invalidates the cache entries in the WebSphere Application Server dynamic cache since this scheduler job and cache instance both execute on same WAS instance. However, with the introduction of Commerce Remote Store (CRS) in v9, the Cache Invalidation strategy has changed. For stores running on the Store server, additional setup is required by leveraging a third-party messaging system to send out cache invalidation requests and triggering the invalidation of data cache objects on the Store server. Apache Kafka is currently the supported messaging middleware. In this post we will use the kafka-zookeeper messaging system to handle cache invalidations.
Cache Invalidation between WC (TS) and Store Server
Here is the overall picture. When a data change event is triggered, the WCS producer will send message(s) to the Kafka broker and mark down the change. And for Kafka consumers, they will continually consult to the Zookeeper, and as they detect changes, they will invalidate the cache accordingly.
Cache Invalidation between WC (TS) Servers
Cache Invalidation between WC (TS) Servers is similar to the above although the TS server will be playing role of message producer and consumer. However, TS to TS messages flow through a different topic from the WC to CRS.
TS to TS topic is suffixed with “PeerCacheInvalidation” for example "WCDemoPeerCacheInvalidation" .
TS to CRS topic is sufficed with “CacheInvalidation” for example "WCDemoCacheInvalidation".
For more details on Kafka Topics, refer to Kafka documentation
Note: TS to TS invalidation is available in WebSphere Commerce Version 22.214.171.124+ with additional configuration. Refer to the 'Additional Configuration to enable TS to TS invalidations' section.
Cache Invalidation for Search Servers
The Cache Invalidation strategy for the search server is same as v7, v8. The Search runtime checks for pending invalidations (by querying the CACHEIVL table) before executing every request. Invalidations are only processed for a maximum of 1 second. If the job completes processing all invalidations, then the server won't check again for 30 seconds. If after 1 second there are still pending invalidations, the invalidation work continues upon the next request execution. Fore more details see: Data Cache in the Search server
The above is just single, stand-alone server setup, but it's possible to also have multiple Store and WebSphere Commerce servers. Additionally, for resiliency we can have a kafka-zookeeper servers cluster comprised of with multiple broker/zookeeper nodes.
Create Kafka/Zookeeper Messaging Service
Update the docker-compose YAML file with following services:
#Kafka Service - Image is based on https://hub.docker.com/r/ches/kafka/
Note: Zookeeper/Kafka can be installed manually, but in this blog we will use docker images.
1. Use run engine command to set Kafka/Zookeeper configuration and build custom docker images. For more details, refer to spiuser example here.
Alternatively you can customize the docker startup logic or use consul/vault. For more details refer to: Docker container start up logic.
FROM <docker registry>/commerce/ts-app:latest
FROM <docker registry>/commerce/crs-app
Use Docker Build to create new images with above configuration.
2. Deploy v9 stack using the custom docker images.
docker-compose -f YAML_file up -d
1. Update JNDI on CRS server.
<jndiEntry jndiName="com.ibm.commerce.foundation.server.services.zookeeper.hostnameport" value="localhost:2181"/>
2. Update K/Z configuration on WC server by modifying foundation component file.
<!-- value to kafka servers connection string -->
Additional Configuration to enable TS to TS invalidations
1. Update default CrossTransactionCache setting to add invalidationJobInterval property within the InstanceProperties tag inside wc-server.xml file.
<CrossTransactionCache ... invalidationJobInterval="5">
The jobinterval should be greater than zero.
WebSphere Commerce (Transaction) Server
Look for following trace in the trace:
com.ibm.commerce.dynacache.commands.MessageSender MessageSender(String topic) ENTRY
com.ibm.commerce.dynacache.commands.MessageSender sendMessage(final String key, final String msg) ENTRY <cache_instance> <dataid>
WebSphere Commerce Remote Store (CRS) Server
In messages.log (server startup) look for:
com.ibm.commerce.component.cache.invalidation.ConsumerThread I consumer configration properties is <all connection properties>
During runtime for valid consumable messages look for:
com.ibm.commerce.component.cache.invalidation.DCConsumerThread action(ConsumerRecord<String, String> record) Time: <timestamp>Thread <threadId>: <message>
For invalid message following warning comes up with exception stack:
com.ibm.commerce.component.cache.invalidation.DCConsumerThread action(ConsumerRecord<String, String> record) consuming the invalidation message has exception