Global cache sample: Configuring for high availability (multi-instance)

Use this sample to create a global cache configuration involving three integration servers, two of which host both a catalog server and a container server. The other integration server hosts only a container server, and must be used as part of a multi-instance integration node.

Before you begin

Read the task topic Configuring the embedded global cache.
For concept information, see Data caching overview.
When you are using the global cache, increase the JVM heap size for integration servers that are hosting cache components. For more information, see the Planning environment capacity section of the WebSphere Extreme Scale product documentation online.

About this task

You can copy and paste the content of the global cache section of configuration files to replace the existing content of the GlobalCache section of your server.conf.yaml files and then modify them as required. The GlobalCache section is a sub-section of the ResourceManagers section in the server.conf.yaml files.

This sample shows how to configure a global cache to be used by three integration servers: two integration servers host both a catalog server and a container server, while the other integration server hosts only a container server, and must be used as part of a multi-instance integration node. In this scenario, a multi-instance integration node cannot be used to host an integration server with a catalog server.

Diagram showing a global cache to be used by three integration servers: two integration servers host both a catalog server and a container server, while the other integration server hosts only a container server, and must be used as part of a multi-instance integration node.

This scenario is equivalent to the policy_multi_instance cache policy XML file in previous versions of IBM Integration Bus and WebSphere Message Broker. It covers a similar multi-instance integration node configuration as the XML file.

For information about other samples, which show how to configure global cache for different scenarios, see the following topics:

Procedure

Complete the following steps to create a global cache configuration involving three integration servers, two of which host both a catalog server and a container server, while the other integration server hosts only a container server, and must be used as part of a multi-instance integration node:

Create two integration servers called integrationServer1 and integrationServer2.
Open the configuration files for your integration servers (integrationServer1\server.conf.yaml and integrationServer2\server.conf.yaml) by using a YAML editor.

You can edit the file by using the built-in YAML editor that is provided in the IBM App Connect Enterprise Toolkit, either by double-clicking the file in the Application Development view or by right-clicking the file and selecting Open with > YAML editor. If you choose to edit the file by using a plain text editor, ensure that you do not include any tab characters (which are not valid in YAML) and use a YAML validation tool to validate the contents of your file.

Copy and paste the content of the GlobalCache section from the supplied integrationServer1.server.conf.yaml and integrationServer2.server.conf.yaml sample files in the install_directory\server\samples\globalcache\ha_multi_instance folder, to replace the existing content of the GlobalCache sections of your corresponding integrationServer1\server.conf.yaml and integrationServer2\server.conf.yaml files.

The GlobalCache section is a sub-section of the ResourceManagers section in the server.conf.yaml file.

integrationServer1.server.conf.yaml:

# Integration server configuration file for use as global cache container
# Provides an equivalent to the "Muti-Instance" Cache Policy XML configuration in previous versions of IBM Integration Bus and Websphere Message Broker
# Uses two integration servers (1 and 2) that are both a catalog server and a container server
# It also uses an integration server (3) which is a container server; this integration server must be part of a multi-instance integration node
# A multi-instance integration node cannot be used to host an integration server with a catalog server
#
# General notes :
#  - Integration Server will load server.conf.yaml from directory set via --work-dir
#  - To ensure valid YAML avoid any use of TAB characters
#  - File paths may be taken as absolute, or relative to the integration server's work directory
#

  GlobalCache:
    cacheOn: true                                               # Set to true to enable Global Cache functionality
                                                                # When using Global Cache it is advisable to change your jvmMinHeapSize and jvmMaxHeapSize depending on 
                                                                # the number of live objects in the heap, complexity of live objects in the heap and number of available cores.
                                                                # see https://www.ibm.com/support/knowledgecenter/SSTVLU_8.6.0/com.ibm.websphere.extremescale.doc/cxsjvmtune.html 
    cacheServerName: 'MyCatalogServer1'                         # The name of this cache server component (a cache server component can be a catalog and/or a container); it must be unique in your global cache system
    catalogServiceEndPoints: 'server1.mycompany.com:2800,server2.mycompany.com:2800'
                                                                # Comma-separated list of hostnames and ports for the catalog servers to use, e.g. 'localhost:2800'
    catalogDomainName: 'WMB_MyCacheDomain'                      # Name of the shared global cache domain; this value will be shared by all catalog servers in the same domain
    catalogClusterEndPoints: 'MyCatalogServer1:server1.mycompany.com:2803:2801,MyCatalogServer2:server2.mycompany.com:2803:2801'
                                                                # Comma-separated list of catalog server connection details in the format 'cacheServerName:catalogCacheServerHost:HAPort:clientPort'
                                                                # If this is a catalog server, cacheServerName should match the value above, and if not, it will be the value used on the integration server hosting it
                                                                # The list should be in the same order for all catalog and container servers which are interacting together in the same domain
    enableCatalogService: true                                  # Set to true to launch a catalog service cache server component in this integration server
    enableContainerService: true                                # Set to true to launch a container service cache server component in this integration server
    enableJMX: true                                             # Allow admin access to this container service via JMX 
    listenerHost: 'server1.mycompany.com'                       # Comma-separated list of hostnames for this cacheServer component, e.g. 'localhost,myserver.mycompany.com'
    listenerPort: 2800                                          # Port number this cache server listens on; it must be unique on this machine
                                                                # Four consecutive ports are assigned, e.g. 2800 for catalogCacheServerListenerPort, 2801 for clientPort, 2802 for JMXServicePort, 2803 for HAPort

    #deploymentPolicyCustomFile: ''                             # Override the deployment policy file (default is <install directory>/server/cachesupport/config/deployment.xml)
    #objectGridCustomFile: ''                                   # Override the ObjectGrid file (default is <install directory>/server/cachesupport/config/objectgrid_xio.xml)
    #overrideTraceSpec: ''                                      # Set a trace level for the cache server components, e.g. ObjectGrid*=event=enabled
    #clientsDefaultToSSL: ''                                    # Set to true to enable SSL for any client connections to the cache servers
    #sslAlias: ''                                               # SSL Alias to use for the cache server components
    #sslProtocol: ''                                            # SSL Protocol to use for SSL connections eg. default is "TLSv1.2"

integrationServer2.server.conf.yaml:

# Integration server configuration file for use as global cache container
# Provides an equivalent to the "Muti-Instance" Cache Policy XML configuration in previous versions of IBM Integration Bus and Websphere Message Broker
# Uses two integration servers (1 and 2) that are both a catalog server and a container server
# It also uses an integration server (3) which is a container server; this integration server must be part of a multi-instance integration node
# A multi-instance integration node cannot be used to host an integration server with a catalog server
#
# General notes :
#  - Integration Server will load server.conf.yaml from directory set via --work-dir
#  - To ensure valid YAML avoid any use of TAB characters
#  - File paths may be taken as absolute, or relative to the integration server's work directory
#

  GlobalCache:
    cacheOn: true                                               # Set to true to enable Global Cache functionality
                                                                # When using Global Cache it is advisable to change your jvmMinHeapSize and jvmMaxHeapSize depending on 
                                                                # the number of live objects in the heap, complexity of live objects in the heap and number of available cores.
                                                                # see https://www.ibm.com/support/knowledgecenter/SSTVLU_8.6.0/com.ibm.websphere.extremescale.doc/cxsjvmtune.html 
    cacheServerName: 'MyCatalogServer2'                         # The name of this cache server component (a cache server component can be a catalog and/or a container); it must be unique in your global cache system
    catalogServiceEndPoints: 'server1.mycompany.com:2800,server2.mycompany.com:2800'
                                                                # Comma-separated list of hostnames and ports for the catalog servers to use, e.g. 'localhost:2800'
    catalogDomainName: 'WMB_MyCacheDomain'                      # Name of the shared global cache domain; this value will be shared by all catalog servers in the same domain
    catalogClusterEndPoints: 'MyCatalogServer1:server1.mycompany.com:2803:2801,MyCatalogServer2:server2.mycompany.com:2803:2801'
                                                                # Comma-separated list of catalog server connection details in the format 'cacheServerName:catalogCacheServerHost:HAPort:clientPort'
                                                                # If this is a catalog server, cacheServerName should match the value above, and if not, it will be the value used on the integration server hosting it
                                                                # The list should be in the same order for all catalog and container servers which are interacting together in the same domain
    enableCatalogService: true                                  # Set to true to launch a catalog service cache server component in this integration server
    enableContainerService: true                                # Set to true to launch a container service cache server component in this integration server
    enableJMX: true                                             # Allow admin access to this container service via JMX 
    listenerHost: 'server2.mycompany.com'                       # Comma-separated list of hostnames for this cacheServer component, e.g. 'localhost,myserver.mycompany.com'
    listenerPort: 2800                                          # Port number this cache server listens on; it must be unique on this machine
                                                                # Four consecutive ports are assigned, e.g. 2800 for catalogCacheServerListenerPort, 2801 for clientPort, 2802 for JMXServicePort, 2803 for HAPort

    #deploymentPolicyCustomFile: ''                             # Override the deployment policy file (default is <install directory>/server/cachesupport/config/deployment.xml)
    #objectGridCustomFile: ''                                   # Override the ObjectGrid file (default is <install directory>/server/cachesupport/config/objectgrid_xio.xml)
    #overrideTraceSpec: ''                                      # Set a trace level for the cache server components, e.g. ObjectGrid*=event=enabled   
    #clientsDefaultToSSL: false                                 # Set to true to enable SSL for any client connections to the cache servers
    #sslAlias: ''                                               # SSL Alias to use for the cache server component 
    #sslProtocol: ''                                            # SSL Protocol to use for SSL connections eg. default is "TLSv1.2"

Create a multi-instance integration node called MultiInstanceNode on two servers, as described in Creating a multi-instance integration node.
Create an integration server called integrationServer3 under the multi-instance integration node.

Copy and paste the content of the GlobalCache section from the supplied integrationServer3.server.conf.yaml sample file in the samples\globalcache\multi_instance folder, to replace the existing content of the GlobalCache section in the SharedWorkPath\components\INODE\servers\IntegrationServer3\server.conf.yaml file.

The GlobalCache section is a sub-section of the ResourceManagers section in the server.conf.yaml file.

integrationServer3.server.conf.yaml:

# Integration server configuration file for use as global cache container
# Provides an equivalent to the "Muti-Instance" Cache Policy XML configuration in previous versions of IBM Integration Bus and Websphere Message Broker
# Uses two integration servers (1 and 2) that are both a catalog server and a container server
# It also uses an integration server (3) which is a container server; this integration server must be part of a multi-instance integration node
# A multi-instance integration node cannot be used to host an integration server with a catalog server
#
# General notes :
#  - Integration Server will load server.conf.yaml from directory set via --work-dir
#  - To ensure valid YAML avoid any use of TAB characters
#  - File paths may be taken as absolute, or relative to the integration server's work directory
#

  GlobalCache:
    cacheOn: true                                               # Set to true to enable Global Cache functionality
                                                                # When using Global Cache it is advisable to change your jvmMinHeapSize and jvmMaxHeapSize depending on 
                                                                # the number of live objects in the heap, complexity of live objects in the heap and number of available cores.
                                                                # see https://www.ibm.com/support/knowledgecenter/SSTVLU_8.6.0/com.ibm.websphere.extremescale.doc/cxsjvmtune.html 
    cacheServerName: 'MyContainerServer3'                       # The name of this cache server component (a cache server component can be a catalog and/or a container); it must be unique in your global cache system
    catalogServiceEndPoints: 'server1.mycompany.com:2800,server2.mycompany.com:2800'
                                                                # Comma-separated list of hostnames and ports for the catalog servers to use, e.g. 'localhost:2800'
    catalogDomainName: 'WMB_MyCacheDomain'                      # Name of the shared global cache domain; this value will be shared by all catalog servers in the same domain
    catalogClusterEndPoints: 'MyCatalogServer1:server1.mycompany.com:2803:2801,MyCatalogServer2:server2.mycompany.com:2803:2801'
                                                                # Comma-separated list of catalog server connection details in the format 'cacheServerName:catalogCacheServerHost:HAPort:clientPort'
                                                                # If this is a catalog server, cacheServerName should match the value above, and if not, it will be the value used on the integration server hosting it
                                                                # The list should be in the same order for all catalog and container servers which are interacting together in the same domain
    enableCatalogService: false                                 # Set to true to launch a catalog service cache server component in this integration server
    enableContainerService: true                                # Set to true to launch a container service cache server component in this integration server
    enableJMX: true                                             # Allow admin access to this container service via JMX 
    listenerHost: 'server1.mycompany.com,server2.mycompany.com' # Comma-separated list of hostnames for this cacheServer component, e.g. 'localhost,myserver.mycompany.com'
    listenerPort: 2804                                          # Port number this cache server listens on; it must be unique on this machine
                                                                # Four consecutive ports are assigned, e.g. 2804 for catalogCacheServerListenerPort, 2805 for clientPort, 2806 for JMXServicePort, 2807 for HAPort

    #deploymentPolicyCustomFile: ''                             # Override the deployment policy file (default is <install directory>/server/cachesupport/config/deployment.xml)
    #objectGridCustomFile: ''                                   # Override the ObjectGrid file (default is <install directory>/server/cachesupport/config/objectgrid_xio.xml)
    #overrideTraceSpec: ''                                      # Set a trace level for the cache server components, e.g. ObjectGrid*=event=enabled   
    #clientsDefaultToSSL: false                                 # Set to true to enable SSL for any client connections to the cache servers
    #sslAlias: ''                                               # SSL Alias to use for the cache server component
    #sslProtocol: ''                                            # SSL Protocol to use for SSL connections eg. default is "TLSv1.2"

Change server1.mycompany.com and server2.mycompany.com in the files to match the host names of the servers hosting the integration nodes.
Restart the integration servers for the changes to take effect.
When they are restarted, IntegrationServer1 and IntegrationServer2 will be both catalog servers and container servers. IntegrationServer3 will be a container server.

What to do next

If you have only one catalog server and you stop it, the cache becomes unavailable and your data is lost. However, if you have multiple catalog servers, one of them can be stopped and restarted without losing data; your data is lost only if all catalog servers are stopped.