Creating a two-node cluster

Learn how to create a two-node cluster and the errors that might occur.

Before you begin to create a cluster, it is important to check for a correct networking setup of your cluster nodes:

IP, netmask, and broadcast addresses must be consistent on each cluster node.
Name resolution is correct and DNS entries are consistent, or each local /etc/hosts file contains entries for all the cluster nodes. Long and short host names are handled case-sensitive in RSCT commands like preprpnode and mkrpdomain, so ensure their correct spellings as host names and in the /etc/hosts entries. It might be helpful to use all lowercase or uppercase letters.
Remove extraneous entries that refer to the same host name from the /etc/hosts table, for instance entries like 127.0.0.2 my_hostname. Such entries might be created during the installation of the operating system, but they conflict with the cluster infrastructure.
Do not define more than one network interface to the same subnet on each node.
All firewalls, including local firewalls, must allow ICMP pings and IP-traffic between the cluster nodes through the following port numbers:
- 12347/udp for the RSCT cthats subsystem.
- 12348/udp for the RSCT cthags subsystem.
- 657/udp for the RSCT rmc subsystem.
- 657/tcp for the RSCT rmc subsystem.

For further instructions and suggestions to set up a networking between cluster nodes, see Using network interfaces .

Create a two-node cluster:

Log on to each node in the cluster:
- Open a shell on each node and log on as root. You might also use a non-root user ID with the appropriate rights to use the sudo command.
- Set and export the environment variable CT_MANAGEMENT_SCOPE to 2 on each node. Since this environment variable must always be set when you execute RSCT and System Automation for Multiplatforms commands, set the variable permanently. For instance by setting the variable in /etc/profile. The CT_MANAGEMENT_SCOPE variable configures that all RSCT and System Automation for Multiplatforms commands run either local (1) or in a cluster (2) scope.
- Run the preprpnode command on all nodes to allow communication between the cluster nodes.
```
preprpnode node01 node02
```
You are now ready to create a cluster with the name SA_Domain containing the nodes node01 and node02. The command must be entered once from any of the two nodes:
```
mkrpdomain SA_Domain node01 node02 
```
When you create RSCT peer domains (clusters) with the mkrpdomain command, the allowed characters for the peer domain name are limited to: A-Z, a-z, 0-9,. (Period), and _ (underscore). The domain name is handled case-sensitive.

To look up the status of the cluster SA_Domain, enter the lsrpdomain command:
```
lsrpdomain
```
Output:
```
Name       OpState RSCTActiveVersion MixedVersions TSPort GSPort
SA_Domain  Offline 3.1.5.3           No            12347  12348
```
The cluster is defined but is offline because it is not started yet.
You can use the -k flag to set the cluster shared secret key (CSSK). The CSSK is used by the RSCT components to authenticate the control messages sent between nodes within the peer domain. By default, the CSSK is disabled (that is, set to CSSKTYPE_None). To enable message authentication, use a CSSK value such as CSSKTYPE_DES_MD5 with the -k flag. Enabling message authentication affects performance. The complexity of the encryption algorithm determines this effect.
Message authentication also requires that the time-of-day clocks (TODs) of the nodes in the peer domain are synchronized — according to the system time — to within 2 minutes of each other. When the nodes' TODs are synchronized across the peer domain, this function helps to defend against message replay attacks. If the nodes' TODs are not synchronized to within 2 minutes of each other, messages that are passed between a sending node and a receiving node with a time difference that is longer than 2 minutes are discarded.

When message authentication is enabled by using the -k flag, a key refresh interval can be specified by using the -r flag. By default, the key is refreshed daily.

For more information about setting up and managing CSSK settings, see the Administering RSCT guide.
To use more secure algorithms, a node or cluster can be migrated to be compliant with the National Institute of Standards and Technology (NIST) SP800-131a compliance.

To use the nist_sp800_131a NIST compliance mode for any service that uses security mechanisms for authentication and secure communication, the service must use minimum key strengths.

After the node is made complaint with the nist_sp800_131a NIST compliance mode, the cluster security services can authenticate sessions from the nodes that are using symmetric or asymmetric keys, which are compliant with the compliance specification.

A stand-alone peer domain can be created to operate in the nist_sp800_131a mode, or migrated to be compliant by running an action while the domain is online.

The mkrpdomain command provides the -C option to specify the compliance mode of the domain that is created and the -k option can be used to specify the cluster key type that is compliant with the requested security mode.
To migrate a domain to the nist_sp800_131a mode, run the following command:
```
> runact -c IBM.PeerDomain ChangeSecurityMode Mode="nist_sp800_131a" CSSKType="key_type"
```
For more detailed information about enabling the nist_sp800_131a security mode in a peer domain, see the section “Configuring security compliance mode” of the Administering RSCT guide.
Issue the startrpdomain command to bring the cluster online.
```
startrpdomain SA_Domain
```
When you run the lsrpdomain command again, the cluster might be displayed with OpState Pending Online while the start is still in progress:
```
Name       OpState        RSCTActiveVersion MixedVersions TSPort GSPort
SA_Domain  Pending online 3.1.5.3           No            12347  12348
```
After a short time the cluster start finished and the lsprdomain command displays the cluster as Online:
```
Name       OpState RSCTActiveVersion MixedVersions TSPort GSPort
SA_Domain  Online  3.1.5.3           No            12347 12348
```

Common errors that might occur during cluster creation

You can receive an error message like the following when you issue the mkrpdomain command:
```
2632-044 The domain cannot be created due to the following
            errors that were detected while harvesting information from the target
            nodes:node1: 2632-068 This node has the same internal identifier as node02
            and cannot be included in the domain definition.
```
This error most often occurs if you use cloned operating system images.

In this case, the cluster configuration must be reset. Solve such problems by running the following command on the node that is specified in the error message. This resets the RSCT node ID:
```
/usr/sbin/rsct/install/bin/recfgct
```
Then, you must repeat the cluster creation steps again starting with the preprpnode commands.

You can get an error message like:

2632-044 The domain cannot be created due to the following
            errors that were detected while harvesting information from the target nodes:
            node1: 2610-418 Permission is denied to access the resources or resource
            class specified in this command.

Check your host name resolution. Make sure that all entries for each cluster node, name server, or both are identical in all local /etc/hosts files.

You can get an error message like:
```
2612-022 A session could not be established with the RMC
            daemon on <node_name>.
```
This error most often occurs when a firewall is blocking communication between the cluster nodes. To resolve the problem, enable the following ports in the firewall software:
```
rmc port 657 TCP IN/OUT Source Port Range: ephemeral port
rmc port 657 UDP IN/OUT Source Port Range: ephemeral port
cthats port 12347 UDP IN/OUT Source Port Range: 1024 - 65535
cthags port 12348 UDP IN/OUT Source Port Range: 1024 - 65535
```
All firewall rules must allow BROADCAST packets to go through the cthats port. For more information about firewall and network problems, see IBM RSCT Administration Guide, Appendix A. RSCT network considerations.