Network

Edit online

A high-speed network connection between the two Db2® Mirror nodes is required.

The internet address of the TCP/IP interface associated with the line description of a RoCE adapter port is used to identify each physical RDMA link. Each RDMA link is assigned to all Db2 Mirror Network Redundancy Groups (NRGs) by default.

Note: An Ethernet line description and a TCP/IP interface for each RoCE adapter port must be manually created on the setup source node before beginning to configure Db2 Mirror. Routed links, which have IP addresses in different subnets, also require a TCP/IP route to be manually created on the setup source node.
The following five NRGs are used for Db2 Mirror.
  1. Db2 Mirror Environment Manager
  2. Database Replication
  3. System Object Replication
  4. IFS Replication
  5. Resynchronization

The five NRGs allow you to have control over which network resources are used for various Db2 Mirror activities. For more information about changing the default configuration of your NRGs, see Managing NRGs.

The following figure shows four RDMA links shared across the five Db2 Mirror NRGs.

Figure 1. Db2 Mirror NRGs
Db2 Mirror NRGs
Each RDMA link is configured to use a particular RDMA protocol. The following RoCE protocols are supported:
  • RoCE v1: RoCE version 1 is a non-encrypted and non-routable RDMA protocol. It transfers data using the Ethernet link layer protocol with Ethertype 0x8915 which allows communication between two hosts within the same Ethernet broadcast domain. The ports of RoCE adapters can be cabled directly together or connected by two cables and a single switch, which provides a maximum distance of 200 meters. Adapter FC EC78 and LR Transceivers (NVIDIA MMA1L10-CR Optical Transceiver 100GbE QSFP28 LC-LC LR4) can support up to 10 kilometers. Communicating host IP addresses must be in the same Ethernet broadcast domain and in the same subnet. This type of link is available only if non-encrypted RDMA is allowed in your environment.
  • RoCE v2: RoCE version 2 is a non-encrypted and routable RDMA protocol. It is an internet layer protocol which enables packets to be routed. The ports of RoCE adapters can be connected through multiple switches or routers, up to a maximum distance of 10 kilometers. Data is not encrypted and is transferred using the UDP protocol and UDP destination port number 4791. This type of link is available only if non-encrypted RDMA is allowed in your environment.
  • Encrypted RoCE v2: Encrypted RoCE version 2 is an encrypted and routable RDMA protocol. RoCE version 2 with IPsec encryption is an internet layer protocol which enables packets to be routed securely by using IPsec protocols to provide data authentication, integrity, and confidentiality. The ports of RoCE adapters can be connected through multiple switches or routers, up to a maximum distance of 10 kilometers. A VPN connection must be configured for each RDMA link endpoint with data transforms (encryption protocols) supported by the adapter. A VPN connection must be available for the RDMA link to start. If the VPN connection is ended, the RDMA link is also ended. Data is encrypted using IPsec and transferred using the UDP protocol and UDP destination port number 4791.

Encrypted RDMA

Db2 Mirror requires encrypted RDMA protocols to be used by default. To allow Db2 Mirror to use non-encrypted RDMA protocols, your security administrator must change the Db2 Mirror encrypted RDMA setting to Not Required.

A VPN connection for RDMA traffic is required for an encrypted RoCE v2 link to start. Each link is required to have a VPN connection configured for UDP port 4791 traffic between the source and target IP addresses.

The data policy of the VPN connection must use settings supported by the RoCE adapter so the encryption can be offloaded to the adapter. For example, the 2CFA adapter requires the data policy to use Encapsulating Security Payload (ESP) with AES-GCM encryption.

A VPN connection used for RDMA data traffic cannot be shared with non-RDMA TCP/IP traffic. After an encrypted RoCE v2 link starts, all other types of traffic that tries to use the same VPN connection is blocked.

You must use policy filter rules to allow UDP traffic using port 3994 which is used for link establishment. This can be achieved in a variety of ways. For example, create a policy filter rule that permits all traffic that does not match any other policy filter rules. Or create a policy filter rule and VPN connection specifically for UDP port 3994 traffic.
Note: The policy filter rules must be configured in such a way so UDP port 3994 traffic and UDP port 4791 traffic will not use the same VPN connection. Creating policy filter rules or VPN connections for traffic other than UDP port 3994 and 4791 is optional.

For more information about configuring policy filter rules for VPN connections, see Configuring VPN packet rules.

The Db2 Mirror GUI can be used to configure VPN connections for encrypted RoCE v2 links. Any VPN connections, Internet Key Exchange (IKE) policies, data policies, or policy filters that are created by Db2 Mirror are created using the attributes and values listed in the table below.
Table 1. Encrypted RoCE v2 VPN Connection Attributes and Values
  Attribute Value
VPN Connection VPN name User provided VPN name
Local address Source IP address
Remote address Copy IP address
Remote key server Copy IP address
Start on-demand Yes
Protect service
Local port 4791
Remote port 4791
Protocol UDP
IKE Policy (pre-shared key) Remote server Copy IP address
Local key server Source IP address
Pre-shared key User provided pre-shared key
Hash algorithm SHA
PRF algorithm AES-XCBC-MAC
Encryption algorithm AES-CBC
Key size 16 bytes
Diffie-Hellman Group 1
Session life 24 hours
IKE Policy (certificate) Remote server Copy IP address
Certificate User provided certificate label
Hash algorithm SHA
PRF algorithm AES-XCBC-MAC
Encryption algorithm AES-CBC
Key size 16 bytes
Diffie-Hellman Group 1
Session life 24 hours
Data Policy Data policy name User provided VPN name
Protocol Encapsulating security payload (ESP)
Encryption algorithm AES-GCM
Key size 16 bytes
Integrity check value (ICV) length 16
Policy Filter for RDMA traffic Action IPSEC
Direction OUTBOUND
Source address Source IP address
Destination address Copy IP address
Protocol UDP
Destination port 4791
Source port 4791
VPN connection User provided VPN name
Policy Filter to permit all non-VPN traffic Action PERMIT
Direction Any
Source Address Any
Destination Address Any
Protocol Any
Destination Port Any
Source Port Any

When using the Db2 Mirror GUI to create a VPN connection for an encrypted RoCE v2 link, you can choose to use either a pre-shared key or certificate for the IKE policy. To use certificates, you first must obtain or create certificates that are associated with the IP addresses of the RDMA links. Then you must add all certificates to the Digital Certificate Manager (DCM) *SYSTEM certificate store on the setup source node. The Db2 Mirror GUI will list the available certificates from the DCM *SYSTEM certificate store and require you to select one when adding new encrypted RoCE v2 links.

Db2 Mirror network planning worksheet

  • What are the performance goals for your network?
  • How much data will be flowing between your Db2 Mirror primary and secondary nodes?
  • How will failover be handled to minimize the impact of the failure of one or more links between your primary and secondary nodes?
  • How many RDMA-capable adapters will you need to meet your goals for mirroring?
    • Each adapter supports up to 2 physical links.
    • Each Network Redundancy Group (NRG) can be configured to specify up to 16 RDMA links.
    • For a given NRG, each RDMA link must map to a unique physical link.
  • How many IP addresses are needed for all the network adapters on both your primary and secondary nodes?
    • An RDMA link is defined by an IP address pair between the primary and secondary nodes.
    • Each IP address should be a static address and be assigned by your network administrator.
    • Each primary and secondary address pair must be in the same subnet.
    • Both IPv4 and IPv6 addresses are supported.
    • An RDMA link can be used by more than one NRG. In other words, a given RDMA link can be used as a default link for data communication for one or more NRGs while also being configured as a failover link for another NRG.
    • Host names are not required for the IP addresses since they are not used when configuring NRGs.
  • How many links are needed in order to achieve the performance goals for mirroring your objects or application data?
    • Do you have high priority applications that need dedicated connections?
    • Can your workloads be balanced across multiple connections?
    • How many address pairs are needed for failover in order to minimize the impact of communication failures?

For more information to help with planning for your network, see Managing NRGs.

TCP/IP ports required for Db2 Mirror GUI

In order to manage the Db2 Mirror environment using the Db2 Mirror GUI, the following ports need to be open: