Planning and prerequisites for your Bare Metal Hosted Control Plane

Choose your configuration

Networking prerequisites

Multi Cluster Engine (MCE)

Step 1: Configure Host Inventory Settings

After the initial setup, configure the host inventory settings. For more information about the host inventory settings, see Red Hat documentation.

Step 2: Create project in host IBM Fusion OpenShift Container Platform cluster for host inventory for Hosted Control Plane cluster to be created

From the OpenShift Container Platform console of IBM Fusion, create a project designated for the host inventory along with other necessary resources to support the Hosted Control Plane cluster.

You can use any name. In this example, cluster name is used for simple identification.

Step 3: Create your infrastructure environment

The next step is to create an infrastructure environment in MCE or using the Hosted Control Plane CLI. This infrastructure environment contains the list of Bare Metal hosts that you can select to create a Bare Metal Hosted Control Plane.

Note: Create one infrastructure environment per hosted control plane cluster. Each Hosted Control Plane can only add hosts from a single infrastructure.

1. In the MCE user interface, go to Infrastructure > Host Inventory.
2. Select Create infrastructure environment.
3. On the Create infrastructure environment page, enter the following details:

   Name

   Name of the infrastructure. No specific name is required.

   Network type

   Select Static IP, Bridges, and Bonds.

   Location

   Label on all hosts

   Labels

   Optional value.

   Pull Secret

   The pull secret must include the following:

   • cloud.openshift.com
   • cp.icr.io
   • quay.io
   • registry.connect.redhat.com
   • registry.redhat.io

   SSH public key

   Generate a private public ssh key pair from bastion host and specify public key in ssh field. It is optional but recommended.

Step 4: Import the hosts by using MCE

With the infrastructure environment created, the next steps are to boot individual servers with an ISO to import them into the host inventory.

Step 4.1: Create the NMStateConfig for each host

The NMStateConfig allows the server to be recognized by the IBM Fusion HCI cluster. For additional information, see Red Hat documentation.

The NMStateConfig varies based on whether static IP assignment or DHCP IP assignment is used. For a fresh IBM Fusion HCI 2.10 installation or an upgrade from 2.9 to 2.10 with the network migration script applied, use the new NMStateConfig. If IBM Fusion HCI is upgraded from version 2.9 to 2.10 without applying the network migration script, retain the old NMStateConfig configuration.

Static IP

New NMStateConfig for static IP assignment:

apiVersion: agent-install.openshift.io/v1beta1
kind: NMStateConfig
metadata:
  labels:
    infraenvs.agent-install.openshift.io: <same as infra name>
  name: <any unique name for host within infra for example fusion-bm-ru8>
  namespace: <same namespace as used for infra>
spec:
  config:
    dns-resolver:
      config:
        server:
          - <DNS server IP>
    interfaces:
      - ipv6:
          enabled: false
        link-aggregation:
          mode: 802.3ad
          options:
            lacp_rate: '1'
            miimon: '140'
            xmit_hash_policy: '1'
          ports:
            - <Bare metal Primary interface’s first port/interface name>
            - <Bare metal Primary interface’s second port/interface name>
        mac-address: <Bare metal Primary interface’s first port’s Mac address>
        mtu: <MTU value>
        name: bond0
        state: up
        type: bond
      - ipv4:
          address:
            - ip: <Bare metal primary interface IP>
              prefix-length: 24
          dhcp: false
          enabled: true
        ipv6:
          enabled: false
        name: <bond name followed by VLAN ID of type OpenShift Customer VLAN present on the cluster for example bond0.VLAN_ID> 
        state: up
        type: vlan
        vlan:
          base-iface: bond0
          id: <VLAN ID of type OpenShift Customer VLAN present on the cluster>
    routes:
      config:
        - destination: 0.0.0.0/0
          next-hop-address: <Gateway IP>
          next-hop-interface: <bond name followed by VLAN ID of type OpenShift Customer VLAN present on the cluster for example bond0.VLAN_ID>
  interfaces:
    - macAddress: <Bare metal Primary interface’s first port’s Mac address>
      name: Bare metal Primary interface’s first port/interface name>
    - macAddress: <Bare metal Primary interface’s second port’s Mac address>
      name: Bare metal Primary interface’s second port/interface name>

Old NMStateConfig for static IP Assignment:

apiVersion: agent-install.openshift.io/v1beta1
kind: NMStateConfig
metadata:
  labels:
    infraenvs.agent-install.openshift.io: <same as infra name>
  name: <any unique name for host within infra for example fusion-bm-ru8>
  namespace: <same namespace as used for infra>
spec:
  config:
    dns-resolver:
      config:
        server:
          - <DNS server IP>
    interfaces:
      - ipv4:
          address:
            - ip: <Bare metal primary interface IP>
              prefix-length: 24
          dhcp: false
          enabled: true
        ipv6:
          enabled: false
        link-aggregation:
          mode: 802.3ad
          options:
            lacp_rate: '1'
            miimon: '140'
            xmit_hash_policy: '1'
          ports:
            - <Bare metal Primary interface’s first port/interface name>
            - <Bare metal Primary interface’s second port/interface name>
        mac-address: <Bare metal Primary interface’s first port’s Mac address>
        mtu: <MTU value>
        name: bond0
        state: up
        type: bond
    routes:
      config:
        - destination: 0.0.0.0/0
          next-hop-address: <Gateway IP>
          next-hop-interface: bond0
  interfaces:
    - macAddress: <Bare metal Primary interface’s first port’s Mac address>
      name: <Bare metal Primary interface’s first port/interface name>
    - macAddress: <Bare metal Primary interface’s second port’s Mac address>
      name: <Bare metal Primary interface’s second port/interface name>

DHCP

New NMStateConfig for DHCP IP assignment:

apiVersion: agent-install.openshift.io/v1beta1
kind: NMStateConfig
metadata:
  labels:
    infraenvs.agent-install.openshift.io: <same as infra name>
  name: <any unique name for host within infra for example fusion-bm-ru8>
  namespace: <same namespace as used for infra>
spec:
  config:
    interfaces:
     - ipv6:
        enabled: false
       link-aggregation:
        mode: 802.3ad
        options:
          lacp_rate: "1"
          miimon: "140"
          xmit_hash_policy: "1"
        ports:
        - <Bare metal Primary interface's first port/interface name>
        - <Bare metal Primary interface's second port/interface name>
      mac-address: <Bare metal Primary interface's first port's Mac address>
      mtu: <MTU value>
      name: bond0
      state: up
      type: bond
     - ipv4:
          dhcp: true
          enabled: true
        ipv6:
          enabled: false
        name: <bond name followed by VLAN ID of type OpenShift Customer VLAN present on the cluster. For example bond0.VLAN_ID> 
        state: up
        type: vlan
        vlan:
          base-iface: bond0
          id: <VLAN ID of type OpenShift Customer VLAN present on the cluster>
  interfaces:
    - macAddress: <Bare metal Primary interface's first port's Mac address>
      name: <Bare metal Primary interface's first port/interface name>
    - macAddress: <Bare metal Primary interface's second port's Mac address>
      name: <Bare metal Primary interface's second port/interface name>

Old NMStateConfig:

apiVersion: agent-install.openshift.io/v1beta1
kind: NMStateConfig
metadata:
  labels:
    infraenvs.agent-install.openshift.io: <same as infra name>
  name: <any unique name for host within infra for example fusion-bm-ru8>
  namespace: <same namespace as used for infra>
spec:
  config:
    interfaces:
    - ipv4:
        dhcp: true
        enabled: true
      ipv6:
        enabled: false
      link-aggregation:
        mode: 802.3ad
        options:
          lacp_rate: "1"
          miimon: "140"
          xmit_hash_policy: "1"
        ports:
        - <Bare metal Primary interface's first port/interface name>
        - <Bare metal Primary interface's second port/interface name>
      mac-address: <Bare metal Primary interface's first port's Mac address>
      mtu: <MTU value>
      name: bond0
      state: up
      type: bond
  interfaces:
    - macAddress: <Bare metal Primary interface's first port's Mac address>
      name: <Bare metal Primary interface's first port/interface name>
    - macAddress: <Bare metal Primary interface's second port's Mac address>
      name: <Bare metal Primary interface's second port/interface name>

The NMStateConfig is dependent on the network environment and must be created in consultation with your network administrators.

Enter the following fields in the NMStateConfig.yaml:

Note: These fields within the NMStateConfig are unique per host.

Field	Description
name	Unique name for each node.
namespace	Namespace is the infraenv namespace.
labels	infraenvs.agent-install.openshift.io: infraNamespace. The infraNamespace must match the name of the infraenvironment into which the host is imported.
ports	The correct ports can be found in the ComputeConfiguration CR. The ComputeConfiguration for the particular node to be imported to MCE. Within the ComputeConfiguration, there is a section “NetworkInterfaces”. Within this section, the interfaceName with “baremetal” has the name of the two interfaces interfaceLeg1 and interfaceLeg2. The values under interfaceLeg1 and interfaceLeg2 are the ports. This guidance is for internal. Consult your network administrator for external.
ipaddress	It is the ipv4 address for that node or server.
interfaces > port: macaddress: and port: macaddress:	You can find these values in the ComputeConfiguration CR. Each port is an entry from the ports. The macaddress is for that port. The macaddress can be ComputeConfiguration for the node by looking for the networkType port: baremetal. Map that macaddress to the networkCards: -slot-#: There will be one matching macaddress and one new one, use those in the NMStateConfig.yaml. Important: change any letters to lower case. Example: ens1f0np1: macaddress: macaddress can be found in the ComputeConfiguration for the node under networkCards: slot-1: macaddress: This guidance is for internal. Consult your network administrator for external.

NMState information is sourced from the switch. For unmanaged servers, you must manually provide the required values.

After the NMStateConfig.yaml is created for a server, it must be added to IBM Fusion HCI cluster using the following commands:

1. Go to the infraenvnamespace namespace.

   oc project infraenvnamespace

2. Apply the YAML.

   oc apply -f nmstate.yaml

Step 4.2: Download the discovery.iso

Use the wget command to download the discovery.iso to the service node or another jumpbox.

Note: Keep this ISO secure as you can use this to add a server to the cluster with sensitive information.

To get the retrieval command do the following:

1. Log into the hub IBM Fusion HCI cluster.
2. In the MCE section, go to Infrastructure > Host inventory.
3. In the infrastructure environment, select the infrastructure you intend to import the host into and select that infrastructure environment.
4. Select Add Hosts > With Discovery ISO. There exists either a URL or a wget command. Use the command or URL to transfer the ISO to your jumpbox.

Step 4.3: Boot the host with the discovery.iso

This task requires you to log into the IMM, mount the ISO image, and reboot the server.

Note: For internal, go through the steps in this section. For external, go through the Red Hat documentation.

Mount the ISO on to the server through the IMM:

1. Get the IPv6 or IPv4 from ComputeConfiguration for that server.
2. Log in to the IMM.
3. Use the USER/Password from secretName secret in the ComputeConfiguration. The defaultUserName and defaultUserPasswrd fields.
4. Select Remote Console tab.
5. In the Remote Console tab, select Launch remote console and Media.
6. In the Media page, go to the Mount media from Client Browser section.
7. Select the discovery.iso image and mount it.
8. Select one virtual media to boot on the next restart.
9. Select the mounted discovery.iso and restart immediately.
10. Close windows and monitor that the remote console is showing the reboot.

Step 5: Accept the host into the MCE Inventory

After the server gets restarted with discovery.iso, accept the host into the host inventory.

1. In the MCE user interface, go to Infrastructure > Host inventory.
2. From the table, select the infrastructure environment where the host got imported.
   Note: In the hosts table, you can see the approve host after the server gets booted up completely. It can take 10-15 minutes. Verify the server before you accept. If the server name is not recognizable, then look at the details to confirm. You can change the host name but ensure it is unique.
3. Accept the host into the inventory.

Step 6: Labeling hosts

After you accept the hosts, add labels to them through the Agent CR. It can be useful to identify a specific cluster or nodepool.

To add the label, do the following steps:

1. Log into the OpenShift console.
2. Go to Administration > CustomResourceDefinitions.
3. Search for Agent of type agent-install-openshift.io.
4. In the instances, find the agent to modify the label.
5. Edit the label of the agent to add a new label.