Planning and prerequisites

To ensure a smooth installation of the IBM Fusion HCI in your data center, review the following network requirements and configuration options. This information is intended for your network administrator.

Initial setup and network connection

The IBM Fusion HCI connects to your data center's network through two high-speed switches. These switches are linked to your core network via a single port channel, which serves as the gateway between the appliance and your network. This connection enables:

Administration of the appliance and OpenShift®
Network traffic in and out of the cluster

Pre-installation network preparation: Before the IBM Fusion HCI appliance installation, your network team must prepare your network. An IBM Systems Service Representative (SSR) performs the initial configuration of the appliance and connect it to your pre-configured network using the information you provide.
Important: Complete the planning worksheets to provide IBM Service Support Representative (SSR) with the IBM Fusion HCI network setup plan.
To download the worksheets, see IBM Storage Fusion HCI Installation worksheets.

When you fill the worksheet, check with your network team about whether the CIDR ranges that you plan to use are free on your network.

Configuring DNS and DHCP

To set up an OpenShift cluster, you need to configure the Domain Name System (DNS) and Dynamic Host Configuration Protocol (DHCP) for each node in the appliance. This involves creating DHCP entries for each MAC address and setting up forward and reverse DNS entries for each IP address.

Before configuring DNS and DHCP, ensure that you have:

Each node should have a pre-configured MAC address and a MAC address for the bootstrap VM.
IBM provides a list of all MAC addresses for configuration purposes.

Follow the instructions to create DNS and DHCP entries:

Create DHCP entries: Create a DHCP entry for each MAC address provided by IBM. This ensures that each node and service node receives an IP address.

Create DNS entries: Create forward and reverse DNS entries for each IP address assigned to the nodes and service nodes.

Here are some key considerations while you create DNS and DHCP entries:

Ensure that your DHCP server can provide infinite leases. This is required to set a permanent IP address for each node.
Set the DHCP expiration time to 4294967295 seconds (as specified by RFC 2131) to enable infinite leases.
Note that in RHEL 8, dhcpd does not provide infinite leases. As an alternative, use dnsmasq to serve dynamic IP addresses with infinite lease times.

To serve dynamic IP addresses with infinite lease times using the provisioner node, it is recommended using dnsmasq instead of dhcpd.

For a detailed guide on setting up DNS and DHCP for IBM Fusion HCI, refer to Setting up DHCP for IBM Fusion HCI.

Configuring static IP

To configure static IP, you need to obtain the following IP addresses:

Gateway IP: The IP address of the gateway.
DNS IP: The IP address of the DNS server.
Service node IP: The IP address of the service node.
For a detailed guide on setting up static IP for IBM Fusion HCI, refer to Setting up Static IP.

These IP addresses are used during the cluster configuration process.

Configuring NTP Server

Ensure that the network administrator has configured an NTP server that is connected to your network.

NTP server requirements

The IBM Fusion HCI requires a connection to an NTP server to coordinate time across the nodes in the OpenShift cluster.
Ensure that the NTP server is accessible on the network where the IBM Fusion HCI is connected.

Providing NTP server information

Provide the IP address of the NTP server in the planning worksheet. This information is required by the IBM SSR to complete the initial configuration of the appliance.
You can provide multiple NTP servers as comma-separated values.

Note: According to Red Hat®, to maintain optimal synchronization in an OpenShift Container Platform cluster, use a minimum of four NTP servers. If four NTPs are not available, then use one or three NTPs servers but never two because it can introduce synchronization issues.

LACP topology

To ensure high availability and redundancy, we recommend configuring the IBM Fusion HCI high-speed switches to connect to the data center switches using link aggregation.

Configuration recommendations

Use both ports 31 and 32 on each high-speed switch to connect to the data center switches.
This configuration creates a total of four links, which are aggregated into a single port channel with four times the bandwidth.

Benefits

Provides redundancy and high availability: if any link or switch fails, traffic is automatically balanced between the remaining links.
Increases bandwidth: the aggregated port channel has four times the bandwidth of a single link.

Switch configuration options

Recommended Configuration: Use two high-speed switches, each connected to the data center switches with two ports (31 and 32).
Alternative Configuration: If you prefer to use only one switch, combine data center switch 1 and data center switch 2 with four ports.

By following these configuration recommendations, you can ensure a reliable and high-bandwidth connection between the IBM Fusion HCI and the data center switches.

It is recommended to use an LACP topology to connect the IBM Fusion HCI switches to the data center switches.

Note: LACP is the preferred choice. Support is not available for non-LACP.

Link Aggregation Control Protocol (LACP) is an IEEE standard that is defined in IEEE 802.3ad to dynamically build an Etherchannel. Here, ISL refers to Inter Switch Link.

For IBM Fusion HCI, the recommended LACP settings are as follows:

Rate setting is Fast
Mode is Active-Active setting

To know more about network cable requirements to connect the appliance switches to the data center switches, see Network cable and transceiver options.

VLAN planning

The IBM Fusion HCI System requires two VLANs for communication between the data center switches and the appliance high-speed switches.

Supported VLAN range: The IBM Fusion HCI System supports VLANs from 1 to 4094.

The following are the required VLANs:

OpenShift customer VLAN

A default VLAN used to access OpenShift.
Requires a name and ID, which must be recorded in the planning worksheet for initial configuration.
For example, OpenShift-Customer (name) and 100 (ID).

Native VLAN

Handles discard traffic.
Typically uses VLAN ID 1, but verify your data centers default ID.
For example: Native (name) and 1 (ID).

Important: The storage VLAN applicable only for the Global Data Platform. Data Foundation does not contain a storage VLAN.

Storage VLAN

Used by the internal 100G storage network.
Default value is 3201, but can be changed during initial configuration.
For example: Storage (name) and 3201 (ID).

Reserved VLAN IDs

The following VLAN IDs are reserved and should not be used:

VLAN 4091 (internal Provisioning network)
VLANs 3725-3999 (internal switches).

VLAN naming conventions

Link names and VLAN IDs must not contain special characters.
Interface names can be up to 15 characters long.
The first character cannot be a number, and dashes (-) are not allowed in the name.

Important considerations

All internal VLANs used within the IBM Fusion HCI must have unique VLAN IDs.
The system does not provide routing between VLANs; routing functions must be provided at the data center core network.
It is recommended to configure the uplink in trunk mode rather than access mode as it allows addition of VLANs to the uplink.

Network for service node

Network overview for service node

The node supports three types of networks, namely provisioning network, Bare Metal network, and Customer DC network. Both provisioning network and Bare Metal network are mandatory and the Customer DC network is optional.

To connect the DC network directly to the service node, connect RJ45 network cable to service node slot 4 port 4. The Port 4 of the quad-port OpenShift Container Platform 1Gb NIC is used to connect the Service Node to the client network. IA Cat5e (or better) Ethernet cable with RJ45 connectors is needed to make the connection from the Service Node to the client switch. IBM does not provide this cable so you must arrange cat5e cable that is commonly available in any data center.

Bare Metal network
Bare Metal node has two 100G ports (with Data Foundation rack), which internally connects to the high-speed switch of the rack. This network is used to reach the OpenShift cluster on the rack. This network is configured by the IBM Fusion HCI install process.
Provisioning network
This has two ports 1G Nic to connect to the provisioning network of the OpenShift. The device comes pre-wired and pre-configured right from the factory.
Customer DC network
This network is used to access service node directly for maintenance and out of band management purposes. It is recommended that you provide an IP, gateway, subnetmask to the SSR at the time of initial setup. Also, check whether you filled in network planing sheet with the right details.

It has 1 GbE Nic that is used to connect to the customer data centre for out of band access.

For more information about the hardware configuration of the service node, see Service node.

Prerequisites for service node

Ensure you meet the following prerequisites for the service node:

A separate VLAN must be available for the Customer DC connectivity to the service node.
Make sure to provide the DHCP-assigned IP address to the bare metal network interface of the service node.
Note: The DHCP and DNS lookup and reverse lookup records are required for service node similar to other OpenShift nodes.
Ensure that the following firewall ports are allowed for the service node:
- Port 22 for SSH access.
- Port 443, 3000, and 3900 for stage 2 user interface (the 443 and 3900 are allocated for IMM console access through port forwarding on service node if needed).

Network for multi-rack HA

Planning and prerequisites for high-availability multi-rack, 3-zone high-availability, and expansion racks:

The provisioning network (VLAN 4091) and OpenShift Bare Metal network must be extended between the racks and must be on the same layer2 domain across all three racks or sites.
The recommended uplink bandwidth requirement is 25G and higher.
The latency must be in the range of less than 10 ms RTT. For more information, see Guidance for OCP Deployments Spanning Multiple Sites.
For Data Foundation multi-rack setup, ensure you meet the following mandatory requirements from lab switches setup:
1. The provisioner VLAN 4091 must be extended on all the intermediate switches between the racks.
2. The OpenShift Container Platform VLAN must be extended on all the intermediate switches between the racks.
Each rack must have the same number of storage nodes. Ensure that the number of NVMe drives are uniform across storage nodes in the multi-rack solution.

Network planning and prerequisites for remote support connection

Remote support is a service that enables IBM Support representative to remotely access your rack and accomplish essential tasks, such as installation, maintenance, and troubleshooting. In order to create a remote support connection, ensure that the additional configuration is required with a customers firewall or proxy, the following table describes required IP and port configuration.

Table 1. Ports, Relay Hosts, and IP Information
Hostname / GEO	IP	Ports
aos.us.ihost.com	72.15.208.234	443
Americas Broker (4.0 Sessions)	150.238.213.135	443
Americas Broker (4.0 Sessions)	72.15.223.60	443
Americas Broker (4.0 Sessions)	72.15.223.62	443

To a remote support connection to work, the customer must allow encrypted TLS outbound traffic for the server 72.15.223.60 on port 443, 150.238.213.135 on port 443 or 72.15.223.62 on port 443.

In order to use geographically specific relay servers and realize improved throughput, the customer should also allow encrypted non-SSL (TLS encryption) outbound traffic to one of the geographically specific relay servers:

72.15.208.234 aos.us.ihost.com (hosted in North America, best for most geographies)
150.238.213.135 aosback.us.ihost.com (hosted in North America, best for most geographies)
72.15.223.60 aosrelay1.us.ihost.com (hosted in North America, best for most geographies)
72.15.223.62 aoshats.us.ihost.com (hosted in North America, best for most geographies)

Remote support connection automatically chooses the broker which provides the best end-to-end performance. All broker servers are available from all geographies, with performance typically better from the server closest to the customer system.

For more information about remote support, see Remote support.

MTU requirement for Backup & Restore Hub and Spoke configuration

This prerequisite is applicable when you create a relationship between two IBM Fusion HCI OpenShift Container Platform clusters. If you plan to make one IBM Fusion HCI as Backup & Restore Hub and another as Spoke, then make sure the following things are met before you install IBM Fusion HCI:

The same MTU (maximum transmission unit) values are set on both clusters.
Confirm with network admins that planned MTU value is supported by your data center infrastructure as well.
All the devices and routers between these two clusters are configured with the same MTU.

Planning for Hosted Control Plane

In the same CIDR range (number of clusters), you must have a set of free available IPs (no DHCP or DNS is required). It is based on the number of Hosted Control Plane clusters that you plan to create in the rack.

Planning and prerequisites for remote mount support

Make sure the following prerequisites are met for remote mount support:

For IBM Storage Scale Erasure Code Edition (ECE)

The storage VLAN (default 3201) must be available and configured on the customer network. If the default storage VLAN 3201 is not available on the customer network, contact IBM Support to change the default VLAN on IBM Fusion HCI.
The default gateway from the storage VLAN must be configured and reachable from IBM Fusion HCI.
The routing must be in place on the customer network from the storage gateway to the external IBM Elastic Spectrum System
The IBM Elastic Spectrum System must also be configured to support MTU 9000 along with other devices en route router, switches. The default MTU of IBM Fusion HCI is 9000.

For Red Hat OpenShift Data Foundation

The routing must be in place on the customer network from the storage gateway to the external IBM Elastic Spectrum System
The IBM Elastic Spectrum System must also be configured to support MTU 9000 along with other devices en route router, switches. The default MTU of IBM Fusion HCI is 9000.

Network definitions

This section provides a comprehensive overview of network definitions, including key terms and concepts.

Important: While you go through these definitions, note that different networking teams use different terminologies. For example, Mellanox or Cisco.

Use LACP aggregation: Whether or not the recommended LACP topology is being used.
Note: LACP is the preferred choice. Use the no aggregation topology only if LACP is not possible as it does not provide the redundancy most clients require.

Link name: The name to assign to the aggregated link created by the recommended LAG topology.

Lag ID: The default LAG ID is 250, and you would only need to customize the ID if it is already in use on your network. A scenario where this might occur is if you have multiple IBM Fusion HCI appliances on the same network.

Spanning tree enabled: If you are not using the recommended LAG topology, specify whether spanning tree is enabled on the network or not.

OpenShift VLAN name: The name to assign to the OpenShift Customer VLAN.

OpenShift VLAN ID: The VLAN ID to assign to the OpenShift Customer VLAN.

VLAN: A virtual LAN (VLAN) is an isolated broadcast domain that is created within a switch. Each VLAN created within a switch is isolated from other VLANs. The network traffic can pass from one VLAN to another by adding a routing device. The routing functions must be provided at the data center core network.

Native VLAN ID: The VLAN ID to assign to the Native VLAN if you aren’t using the default of 1.

Storage VLAN ID: The VLAN ID to assign to the 100G storage network if you aren’t using the default of 3201. You would only normally customize this if you are configuring a Metro Disaster Recovery configuration between two IBM Fusion HCI clusters.

MLAG: A Multi-Chassis Link Aggregation Group (MLAG) allows for multi-system link aggregation and facilitates active-active uplinks of access layer switches. An MLAG with no Spanning Tree configured avoids the wasted bandwidth that is associated with links that are blocked by the spanning tree.; To ensure high availability for the system, IBM Fusion requires MLAG and link aggregation for the switches.

Switch LAG ID: This Switch LAG ID is used by high-speed switches inside the IBM Fusion rack, which is not related to the LACP ID on (external) switch. It is unique for each IBM Fusion rack and can be less than 250.
The high-speed switch (MLAG PAIR) that gets connected to customer switch must have a unique system MAC address (MLAG ID). This MLAG ID is used as source MAC address for traffic that is sourced from MLAG PAIR, such as STP BPDUs. The MLAG ID is internally derived from the Switch LAG ID. The specific Switch LAG ID is used as the last octet for "44:38:39:ff:00:xx".

Layer 2 connections

The connections from all the IBM Fusion high-speed switches to the data center core network and customer management network are all layer 2 connections. IBM Fusion supports the Link Aggregate Control Protocol (LACP) type of layer 2 aggregation.

The following characteristics describe layer 2 connections:

Layer 2 is considered switching and is done at the hardware layer.
Layer 2 is in the same broadcast domain or local network.
Layer 2 finds adjacent partners by MAC address.

Layer 3 connections

IBM Fusion does not participate in any layer 3 routing or firewall functions. These functions are done in the data center core network.

The following characteristics describe layer 3 connections:

Layer 3 is considered routing and is done at the software layer.
Layer 3 knows how to traverse multiple networks (hops).
Layer 3 finds adjacent partners by IP address.

Spanning tree

In the networking stages of the installation for IBM Fusion, an option to enable or disable the Spanning Tree is available. When you enable it, the spanning tree is enabled on the high-speed switches. By default, the option is disabled on these switches. The high-speed switches in the IBM Fusion rack support only rapid spanning-tree (RSTP) modes. It is also compatible with PVST and PVST+. Customer data networks that are running older spanning-tree methods (MST) are not supported.

Note: IBM Fusion sets the spanning-tree bridge priority to 32768 as the default value. To avoid IBM Fusion high-speed switches from becoming the spanning-tree root, the customer switch must have priority less than 32768. For more information, see Spanning Tree and Rapid Spanning Tree - STP.

Port type: Specify whether you are using VLAN trunking or access ports.

Access: An access port provides access to a single VLAN only. Typically, the packets on an access port are raw Ethernet frames (untagged packets).
Trunk: A trunk connection is used to pass traffic from multiple VLANs between two switches. All trunks use the IEEE 802.1Q standard. This link can be a single wire or one of the aggregations methods.

Transceiver: Specify the type of cable that is being used to connect the HCI high-speed switches with your data center switches. See Hardware Compatibility List.

NTP server: The IP address of the NTP server that is used by IBM Fusion HCI.

Aggregate links: Link aggregation is to combine multiple network connections in parallel.
The standard-based negotiation protocol, which is known as IEEE 802.1ax Link Aggregation Control Protocol (LACP), is a way to dynamically build an Etherchannel.