CICS IPIC high availability

The IPIC high availability (HA) capability is supported for IPIC connections between IBM® z/OS® Connect servers and CICS® regions.

This capability ensures resilient access to CICS and balances connections across a cluster of CICS regions. z/OS Communications Server offers two balancing mechanisms, TCP/IP port sharing and Sysplex Distributor. You can configure Sysplex Distributor to prefer local connections. For more information, see Sysplex distributor.
Note: IPIC high availability provides connection high availability by failing over to an alternative CICS region, in the event of a failure. After a connection is established to a CICS region, it persists until the connection is either lost or ended when using the HA reconnection feature. For more information, see HA reconnection.

Workload distribution of requests across selected CICS regions can be achieved using CICS connection groups, see CICS IPIC workload distribution. Requests cannot be distributed over HA connections.

Establishment of an HA connection

Figure 1. IPIC HA connection flows to the generic and specific ports.
This diagram shows the flows in the establishment of an IPIC HA connection.

The CICS regions listen on two end points that are defined by TCPIPSERVICE resources. The generic end point is shared by all regions in the cluster and a specific end point that is used exclusively by a specific region. The IPCONN can be auto-installed, or pre-defined. When using security, ensure that the generic and specific TCPIPServices specify the same security credentials.

A IBM z/OS Connect server connects to a CICS region in the cluster by using the generic port specified in the zosconnect_cicsIpicConnection definition. The connection request on the shared generic port is intercepted by the connection-balancing mechanism and routed to one of the CICS regions in the cluster. The selected CICS region then returns the IP address and port of its specific end point to the IBM z/OS Connect server, and the connection is re-established using those.

HA connection failover

When an IPIC HA connection is lost, or closed because there is no heartbeat response, IBM z/OS Connect automatically attempts to establish a new connection using the shared generic port. This provides a connection failover mechanism in the event that a CICS region becomes unresponsive or fails. The heartbeatInterval attribute on the zosconnect_cicsIpicConnection element sets the time that the connection must be inactive before a heartbeat request is sent to the CICS region. If no response is received from CICS within the interval time, IBM z/OS Connect closes the connection.

HA reconnection

From V3.0.56, IBM z/OS Connect provides a mechanism to automatically reconnect an IPIC HA connection to enable re-balancing of connections. Connections can become unbalanced due to connection failover. A new connection is established before the existing connection is closed, and outstanding requests are allowed to complete. This results in an existing connection being switched to another CICS region while a workload is flowing, without request failures due to the switch process.

This capability is enabled by specifying a reconnect interval using the reconnectInterval attribute on the zosconnect_cicsIpicConnection element. Optionally a preferred CICS region can be specified using the preferredSpecificHost and preferredSpecificPort attributes. Table 1 describes the reconnection behavior dependent on the settings of these attributes.

When the reconnect interval expires for an established connection:
  1. If a preference is specified and this matches the existing connection, IBM z/OS Connect waits for another reconnect interval.
  2. If the existing connection does not match the preference, or no preference is specified, an attempt is made to establish a new IPIC connection, using the shared port. The port sharing or sysplex distributor algorithm selects the best endpoint to keep connections balanced.
  3. If the connection succeeds, a preference is specified, and the newly connected CICS region does not match the preference, then the connection on the shared port is closed and IBM z/OS Connect waits for another reconnect interval.
  4. If the newly connected CICS region matches the preference, or no preference is specified, then the connection on the shared port is closed and an attempt is made to establish the connection on the specific port.
  5. New requests are sent over the new connection.
  6. The previous connection is closed when all outstanding requests have completed.
Note:
  • The preferred specific host and port values are not used to directly establish a connection, they are used only to decide whether a connection should be switched. This ensures that the port sharing technology remains in control of connection balancing.
  • Updates to the preferredSpecificHost and preferredSpecificPort attributes take immediate effect on existing connections, whereas updates to the reconnectInterval attribute take effect only for new connections.
Table 1. Parameter settings for IPIC HA reconnection
reconnectInterval preferredSpecificHost preferredSpecificPort Reconnection behavior
0 - - Reconnection is disabled. This is the default.
A time duration Host name of preferred LPAR
(See Note 1.)		 Port number of preferred CICS region Reconnection is attempted when the time interval expires and the existing connection is not using the preferred host and port. Use this option when using sysplex distributor and a particular CICS region is preferred.
A time duration Host name of preferred LPAR
(See Note 1.)		 - Reconnection is attempted when the time interval expires and the existing connection is not to the preferred host. Use this option when using sysplex distributor and a particular LPAR is preferred.
A time duration - Port number of preferred CICS region Reconnection is attempted when the time interval expires and the existing connection is not using the preferred port. Use this option when using port sharing and a particular CICS region is preferred.
A time duration - - Reconnection is attempted when the time interval expires. Use this option to enable regular connection re-balancing.
Note:
  1. From V3.0.57.0, if the preferred CICS region is on the same LPAR as the IBM z/OS Connect server, you can specify preferredSpecificHost="local"
Check the messages.log file to see when a reconnection occurs. For example, if preferredSpecificHost="9.3.4.123", the following log entries describe the reconnection process:
  1. An initial connection is established: 
    BAQR0680I: CICS connection cicsConn established to host 9.3.4.123 on port 3333 with 100 sessions.
  2. The connection fails: 
    BAQR0664E: Communication failure for CICS connection cicsConn to host 9.3.4.123 on port 3333, reason: java.io.IOException: End of stream.
  3. A connection is established to a non-preferred CICS region: 
    BAQR0680I: CICS connection cicsConn established to host 9.3.5.678 on port 5555 with 100 sessions.
  4. HA reconnection establishes a connection to the preferred CICS region: 
    BAQR0704I: CICS connection cicsConn established to host 9.3.4.123 on port 3333 with 100 sessions, due to reconnection.
  5. The connection to the non-preferred CICS region is closed: 
    BAQR0705I: CICS connection cicsConn to host 9.3.5.678 on port 5555 has closed, due to reconnection.