Node configuration for Multus public network

Configure OpenShift worker and storage nodes to route host traffic to pods on the public network by using NodeNetworkConfigurationPolicy objects with the NMState Operator.

OpenShift worker and storage nodes must be configured to route host traffic to the Pods on the public network through the host public network interface.

The recommended way to configure nodes is using OpenShift NodeNetworkConfigurationPolicy objects. This method can be supported for all OpenShift users regardless of deployment strategy. This method requires the NMState Operator to be installed and enabled. For more information, see Kubernetes NMState Operator.

Each node must obtain an IP address on the ODF public network in the node public network address range. Static IP address management is the only IPAM method that can be supported for any OpenShift cluster. Thus, static management is Fusion Data Foundation supports only the static management method. This requires one NodeNetworkConfigurationPolicy object per host. The template that can be used to configure a host is shown below.

Important: This template below creates an interface called "shim" interface on each host. The shim interface uses the host public network interface (for example, vlan220) as a parent. The static IP is then given to the shim interface and not to the parent. Similarly, routing uses the shim. This is a critical detail. Macvlan disallows the virtual network of connected Pods on any given host from reaching the host directly or through switch hairpin. Without the shim interface, Fusion Data Foundation will not function properly. Do not attempt to set up the Fusion Data Foundation Multus public network without configuring the shim interface.
apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: ceph-public-net-shim-{{NODE_NAME}}
spec:
  nodeSelector:
    node-role.kubernetes.io/worker: ""
    kubernetes.io/hostname: {{NODE_NAME}}
  desiredState:
    interfaces:
      - name: odf-pub-shim
        description: Shim interface to connect to ODF public network
        type: mac-vlan
        state: up
        mac-vlan:
          base-iface: vlan220 # host public network interface
          mode: bridge
          promiscuous: true
        ipv4:
          enabled: true
          dhcp: false
          address:
            - ip: 192.168.252.1 # static IP in node public network range
              prefix-length: 23 # node public network range mask
    routes:
      config:
        - destination: 192.168.240.0/21 # pod public network range
          next-hop-interface: odf-pub-shim

First, follow comments in the template to update the base template for the environment being deployed. Then, for each node, copy the template, and fill in {{NODE_NAME}} and a unique static IP for each node.