switches part of the YAML file depends on the version your system is on.
22.214.171.124 and later
switches: #BEGIN BGP vars: cp4d_asplain: <VALUE> cp4d_network: <VALUE> cp4d_network_vip: <VALUE> hosts: FabSw1a: ansible_host: localhost vrr_ip_addr: <VALUE> cp4d_routerID: 126.96.36.199 isl_peer: 188.8.131.52 bgp_links: link1: swp: <VALUE> neighbor: <VALUE> ip_addr: <VALUE> mtu: 9000 link_speed: 10000 FabSw1b: ansible_host: localhost vrr_ip_addr: <VALUE> cp4d_routerID: 184.108.40.206 isl_peer: 220.127.116.11 bgp_links: link1: swp: <VALUE> neighbor: <VALUE> ip_addr: <VALUE> mtu: 9000 link_speed: 10000 #END BGP #BEGIN L2 switches: hosts: FabSw1a: ansible_host: localhost external_connection_config: external_link1: switch_ports: ['<VALUE>', '<VALUE>'] port_config: mtu: 9000 link_speed: 10000 vlans: ['VALUE'] strict_vlan: <VALUE> name: <VALUE> lacp_link: True lacp_rate: Fast clag_id: 100 partner_switch: 'FabSw1b' #END L2
18.104.22.168 and earlier
switches: hosts: FabSw1a: # Run on the control rather than remotely ansible_host: localhost # Is there an external link here at all? external_connection_enabled: False # If the above is True the following section is filled in external_connection_config: # First link config external_link1: # Ports to put in this link, a list of 'number', 'number' switch_ports: ['XX', 'YY'] # same config for all the values in switch_ports port_config: mtu: 9000 link_speed: 10000 # Which VLANs arriving on this port (and crossing the bridge) vlans: ['VALUE'] # True means we only accept tagged packets from external source # False means we only accept untagged packets from external source strict_vlan: true # Name of this link name: <VALUE> # True indicates there is multiple links in this connection lacp_link: True # LACP Refresh rate. (Fast or Slow) lacp_rate: Fast # CLAG Id for this link, 0 for no clag clag_id: 100 # This switch should have matching cfg (and the same clag_id) # Use False for links only on a single switch partner_switch: 'FabSw1b'
switches section contains two mutually exclusive sections marked
#BEGIN BGP (...) #END BGP and
#BEGIN L2 (...) #END L2. Only one of
these sections must be used, depending on whether your system is configured for Border Gateway
Protocol (BGP). Refer to the section that pertains to your setup:
System configured for BGP
#BEGIN L2 (...) #END L2together with the enclosing comments. Then, provide the following values. Some of these values can be found in the Site Survey provided when the BGP feature was installed by IBM.
- Provide the following values in the main BGP section.
- cp4d_asplain: <VALUE>
- This is the ASN (autonomous system number) that is assigned to Cloud Pak for Data System for BGP to work. This must be different from the ASN assigned to customer switches. A different ASN is required for every Cloud Pak for Data System that utilizes BGP on the network.
- cp4d_network: <VALUE>
- Network assigned to the Cloud Pak for Data System cluster, in CIDR notation, from customer. Minimum /29, typically /26 or more.
- cp4d_network_vip: <VALUE>
- The IP shared between the switches, the gateway used as next hop (default gateway of nodes)
Provide the following values for each of the fabric switches. In a single switch environment, remove the section for
FabSw1b. You might also need to add additional
FabSw1a: vrr_ip_addr: <VALUE> cp4d_routerID: 22.214.171.124 isl_peer: 126.96.36.199 bgp_links: link1: swp: <VALUE> neighbor: <VALUE> ip_addr: <VALUE> mtu: 9000 link_speed: 10000
- vrr_ip_addr: <VALUE>
- Switch specific address, including the
/XXsuffix, inside the
- cp4d_routerID: 188.8.131.52
- Unique identifier. The defaults in the template are 184.108.40.206 and 220.127.116.11. The values must be unique. The addresses 18.104.22.168 through 22.214.171.124 have been reserved for Cloud Pak for Data System router identifiers.
- There must be a link section for each wire between Cloud Pak for Data System and the customer router. The default case is one
link from each Cloud Pak for Data System switch. You might need to
add the extra
linkXsections to match the wiring, one section per link. In a HA setup, the number of links might be different for each switch, but it is not well balanced from the performance perspective.
- swp: <VALUE>
- The port number.
- neighbor: <VALUE>
- Peer to peer link which faces the customer. This is the address on the customer router.
- ip_addr: <VALUE>
- Peer to peer link address on Cloud Pak for Data System end,
/XXsubnet mask (typically /30 or /31).
link1: swp: 46 neighbor: <VALUE> ip_addr: <VALUE> mtu: 9000 link_speed: 10000 link2: swp: 47 neighbor: <VALUE> ip_addr: <VALUE> mtu: 9000 link_speed: 10000 link3: swp: 48 neighbor: <VALUE> ip_addr: <VALUE> mtu: 9000 link_speed: 10000
After you change the values in
switchessection to cover BGP, you also need to change the following sections from the
application_networksection in the yml file:
- application_network_enabled: BGP
- Change the default value
- prefix: <VALUE>
network1: prefixmust match the
- gateway: <VALUE>
network1: gatewayis the
- vlan: <VALUE>
network1: vlanmust be 4080. Cloud Pak for Data System is expecting untagged frames from its peer.
System not configured for BGP
If the system is not configured for BGP, you must remove the section marked
(...) #END BGP together with the enclosing comments. In the remaining section, fill in the
- Ansible hostRunning Ansible on
localhostshould always be the case unless in special conditions for example the fabric switches are not accessible from
- Configure fabric switches connection with customer switchesIf there is no customer switch to be configured (which should be a very rare case), set it to
Falseand quit the whole section. Otherwise set to
- Set the fabric switch ports used to connect to customer switch(es). At least one value must be
Depending on the number of customer switches, there could be one or more links between the fabric switch and customer switch(es). For example, in a single customer switch configuration, a single link is connected to the fabric switch's port 48 from the customer switch.
Another example is 2 links from a pair of HA customer switch connect to 2 ports of the fabric switch, one connects to port 47 and the other connects to port 48.
switch_ports: ['47', '48']
- Switch port configurationSet switch port MTU size and link speed (in mbps)
port_config: mtu: 9000 link_speed: 10000
- Configure customer VLANSet the list of customer VLAN numbers. The VLANs must be separated by a comma (,) and enclosed in square brackets ([ ]).
- Configure strict VLAN option
Strict VLAN option is bidirectional. If set to
True, the fabric switch always sends and only accepts explicitly VLAN tagged Ethernet frames to/from the customer switch(es). Otherwise if
Falseis used, the fabric switch always sends untagged Ethernet frames to the customer switch(es); the fabric switch accepts both explicitly VLAN tagged and untagged Ethernet frames from the customer switch(es).
- Configure interface name on the fabric switch
Any name such as
h0, unless it conflicts with existing names (i.e. e1n1, isl, etc.).
- Configure LACP related link options
The fabric switch can be made up of a single switch or a pair of HA switches.
For single switch case, set
lacp_link: True lacp_rate: Fast clag_id: 0 partner_switch: 'False'
For HA fabric switches case, configure the values as below:
lacp_link: True lacp_rate: Fast clag_id: 100 partner_switch: 'FabSw1b'Note:
- If Cloud Pak for Data System is configured with HA switches,
bonding must be used. Single ports are not supported.
lacp_linkmust be set to
True. Link Aggregation Control Protocol (LACP) 802.3ad is the only supported bonding mode. Switch settings must not be changed in any other files than the YAML file.
clag_idvalue must be unique across all links. If you have only one switch,
clag_id= 0. If you have multiple switches,
clag_idmust have a value that is unique in a range starting at 100. For example,
clag_idof 101, and so on.
- If Cloud Pak for Data System is configured with HA switches, bonding must be used. Single ports are not supported.