Installation worksheet

This worksheet is intended for customers as part of the TDA process to outline the items required to be implemented by the SSR during setup.

Customer task

Customer fills out the desired IP addresses.

SSR task

The SSR uses this information to properly set the desired IP addresses on each canister and the EMS (if applicable).

Note: The management IPs should be on the same subnet as the EMS bridge interface. If you already have an existing EMS, set IP addresses on the same subnet (all must be reachable).
Note: This document does not describe about the high-speed network. Customers must have that in place and ready to fill in (or update) the /etc/hosts file on the EMS node during the software installation. For reference, see Installing and IBM Elastic Storage® System 3000 Quick Deployment Guide. The high-speed interfaces are on a separate switch and should be defined on a different subnet.
  • For instance, if your management interfaces are all 192.168.45.X/24, your high-speed hosts should be any other network. For instance, 172.16.0.0/24. You should use /24 (255.255.255.0), if possible.
Prerequisite (for SSR): You must be connected to the SSR access port of the target canister (top or bottom). See the IBM Storage Scale System 3000 installation (SSR task) for steps required to check the hardware and set the desired IP addresses outlined in this worksheet by the customer.
Figure 1. Front and rear view of IBM Elastic Storage System 3000
Front and rear view of IBM Elastic Storage System 3000
Figure 2. Rear view of ESS 3000
Rear view of ESS 3000
Figure 3. Management ports for each canister
Management ports for each canister
EMS Ethernet (1Gb) port mapping
Note: C10-T2 listed in the figure below would be used as the Container interface.
Figure 4. Port mapping on the back of the EMS
Port mapping on the back of the EMS

The SSR connects to C10-T4 (bottom port) on the 1Gb card to access the server and run the essutils tool.

Figure 5. Rear view of the POWER8 EMS node
Rear view of the POWER8 EMS node
Start of changeFigure 6 shows the EMS server P2P cabling.
Figure 6. EMS server P2P cabling
EMS server cabling diagram
End of change
Start of changeSAS HBA Port Schema Figure 7 shows the SAS HBA port schema.
Figure 7. SAS HBA port schema
SAS HBA port diagram
End of change

SSR / Customer:

  • Ensure that the management switch is in the rack and powered on.
  • Ensure that there is one Ethernet cable from the management switch (if there are VLANs, connect from the xCAT/management VLAN) to port 1 (management port) on each canister as shown in Figure 3.
  • If an EMS node was received with ESS 3000, connect two Ethernet cables from the management switch (if there are VLANs, connect from the xCAT/management VLAN) to port C10-T1 (top port) and to port C10-T2 (second from top).

SSR: General guidance

  • For each ESS 3000 canister in the flow (see Installing), connect your laptop point-to-point to the Service Port as shown in the Figure 2. You need to use this port to access each canister, perform hardware checks, and set the management IP address.
  • For the EMS, connect your laptop point-to-point to C10-T4 and use the instructions in the IBM Storage Scale System 3000 installation (SSR task) to log in, perform the hardware checks, and set the management IP address.

Reference:

  • Canister 1 (top) SSR service port has a default IP address of 10.0.0.102
  • Canister 2 (bottom) SSR service port has a default IP address of 10.0.0.101
  • SSR user ID is ‘essserv1’
  • SSR default password is the ESS 3000 serial number
    • Add ‘B’ for top canister
    • Add ‘A’ for bottom canister
  • Other than the SSR access port, there are no default IP addresses set on any node
  • All management interfaces should be connected to the same VLAN switch
  • Default physical POWER8 EMS ports
    • C10-T1 = Management Interface (will be set by the SSR)
      • Logical port enP3p9s0f0
    • C10-T4 = FSP interface. Default IP of 10.0.0.1. SSR needs to connect laptop to this port when performing the EMS tasks
      • Logical port enP3p9s0f3
  • Start of changeDefault physical POWER9 EMS ports
    • C11-T1 = Management Interface (will be set by the SSR)
      • Logical port enP1p8s0f0
    • C11-T2 = FSP interface. Set by SSR
      • Logical port enP1p8s0f1
    End of change
  • Default physical ESS 3000 ports (per canister) (see Figure 2)
    • Port 1 – Management port (will be set by the SSR)
      • Logical port enp29s0f0
    • Port with a gear symbol (see Figure 2) – Service/Technician port. The SSR will connect a laptop to this port when performing the ESS 3000 tasks. You need to move the cable when working on a different canister (two per ESS 3000 unit)
      • Logical port enp1s0
Note: Start of changeThe worksheet has default IPs filled out in case they were not provided by the customer during the TDA process. Before setting any default IPs have the customer confirm that they are not currently in use by the associated networks.End of change

Start of changeCustomers must fill in the following values so that the SSR is able perform the required networking tasks. If there are more than one Start of changebuilding blocksEnd of change are being installed, you will need to add the corresponding rows.End of change

Recommendations:

  • Keep all management interfaces on 192.168.x.x/24 (netmask 255.255.255.0).
  • Keep all BMC/FSP (HMC1) interfaces on 10.0.0.x/24 (netmask 255.255.255.0).
Note: The EMS has an additional FSP connection at C11-T2, which is visible to the operating system.
Important: All IP addresses must on the same subnet. For example,
  • All management interfaces on 192.168.x.x/24
  • All FSP interfaces on 10.0.0.x/24
ESS 3200/3500 notes:

The ESS 3200 has a single interface for both the BMC and Management interfaces. When mixed with an ESS 5000, you must apply changes to your switch, which would configure a set of ports specific to ESS 3200.

The ESS 3500 has dedicated management (f1) and BMC (f0) interfaces. The management interfaces are connected to the yellow network, while the BMC interfaces are connected to the green trunk ports.

In both ESS 3200 and ESS 3500 proper switch configuration and VLAN tag should be set to route traffic from the BMC to the blue service/FSP network.

ESS 3000 notes:

The ESS 3000 has a single interface per canister used for the management network only (yellow network). There is no BMC connection in ESS 3000.

Campus or remote connection notes:

A POWER9 EMS campus connection must be set prior to deployment (C11-T3). This allows remote access to the EMS and ensures you will not lose a connection when starting the container. Optionally, space is also allocated to set a campus connection on the HMC2 port. This will allow remote access to the FSP which aids the recovery of the node (console/power control) in case of an outage.

The SSR is provided the commands to set the campus interface (C11-T3) and HMC2 port connection to the customer campus connection. If C11-T3 (POWER9 EMS) is not in place or the IP address not provided to the SSR at time of Install Complete may still, consider the task complete. It will be up to the customer/LBS to set this connection before deployment can begin.

ESS 3500 Building-Block 1 (lowest position in frame)
Note: ESS 3500 only applies to ESS 6130 and later.
  IP address Netmask
ESS 3500 canister 1 management interface (left) 192.168.45.30 255.255.255.0
ESS 3500 canister 1 BMC interface (left) 10.0.0.121 255.255.255.0
ESS 3500 canister 2 management interface (right) 192.168.45.31 255.255.255.0
ESS 3500 canister 2 BMC interface (right) 10.0.0.122 255.255.255.0
ESS 3500 Building-Block 2 (higher position in frame)
Note: ESS 3500 only applies to ESS 6130 and later.
  IP address Netmask
ESS 3500 canister 1 management interface (left) 192.168.45.32 255.255.255.0
ESS 3500 canister 1 BMC interface (left) 10.0.0.123 255.255.255.0
ESS 3500 canister 2 management interface (right) 192.168.45.33 255.255.255.0
ESS 3500 canister 2 BMC interface (right) 10.0.0.124 255.255.255.0
The VLAN tag is required to route traffic on the BMC interface to the service network. This interface is connected to the green trunk ports on the switch. This is required only on ESS 3200 and ESS 3500. The default is 101.
ESS Model Tag Default value
ESS 3200/3500 VLAN Tag VLAN TAG 101
ESS 3200 Building-Block 1 (lowest position in frame)
  IP address Netmask
ESS 3200 canister 1 management interface (left) 192.168.45.40 255.255.255.0
ESS 3200 canister 1 BMC interface (left) 10.0.0.131 255.255.255.0
ESS 3200 canister 2 management interface (right) 192.168.45.41 255.255.255.0
ESS 3200 canister 2 BMC interface (right) 10.0.0.132 255.255.255.0
ESS 3200 Building-Block 2 (higher position in frame)
  IP address Netmask
ESS 3200 canister 1 management interface (left) 192.168.45.42 255.255.255.0
ESS 3200 canister 1 BMC interface (left) 10.0.0.133 255.255.255.0
ESS 3200 canister 2 management interface (right) 192.168.45.43 255.255.255.0
ESS 3200 canister 2 BMC interface (right) 10.0.0.134 255.255.255.0
The VLAN tag is required to route traffic on the BMC interface to the service network. This interface is connected to the green trunk ports on the switch. This is required only on ESS 3200 and ESS 3500. The default is 101.
ESS Model Tag Default value
ESS 3200/3500 VLAN Tag VLAN TAG 101
ESS 5000 Building-Block 1 (lowest position in frame)
  IP address Netmask
IO node 1 management Interface (bottom node in building-block) 192.168.45.21 255.255.255.0
IO node 1 FSP (HMC1 port) interface (bottom node in building-block) 10.0.0.101 255.255.255.0
IO node 2 management Interface (top node in building-block) 192.168.45.22 255.255.255.0
IO node 2 FSP (HMC1 port) interface (top node in building-block) 10.0.0.102 255.255.255.0
ESS 5000 Building-Block 2 (upper position in frame)
  IP address Netmask
IO node 1 management interface (bottom node in building-block) 192.168.45.23 255.255.255.0
IO node 1 FSP (HMC1 port) interface (bottom node in building-block) 10.0.0.103 255.255.255.0
IO node 2 management interface (top node in building-block) 192.168.45.24 255.255.255.0
IO node 2 FSP (HMC1 port) interface (top node in building-block) 10.0.0.104 255.255.255.0
ESS 3000 Building-Block 1 (lower position in frame)
  IP address Netmask
ESS 3000 Canister 1 management Interface (left) 192.168.45.60 255.255.255.0
ESS 3000 Canister 2 management Interface (right) 192.168.45.61 255.255.255.0
ESS 3000 Building-Block 2 (higher position in frame)
  IP address Netmask
ESS 3000 Canister 1 management Interface (left) 192.168.45.62 255.255.255.0
ESS 3000 Canister 2 management Interface (right) 192.168.45.63 255.255.255.0
ESS 5000 POWER9 protocol nodes
  IP address Netmask
Protocol node 1 management interface (bottom-most) 192.168.45.50 255.255.255.0
POWER9 protocol node 1 FSP (HMC1 port) interface (bottom-most) 10.0.0.110 255.255.255.0
Protocol node 2 management interface (top) 192.168.45.51 255.255.255.0
POWER9 protocol node 2 FSP (HMC1 port) interface (top) 10.0.0.111 255.255.255.0
Note: If there are additional building-blocks or POWER9 Protocol nodes, please add more tables.
POWER9 EMS (should be only one)
  IP address Netmask
EMS management interface 192.168.45.20 255.255.255.0
EMS FSP (HMC1 port) interface 10.0.0.100 255.255.255.0
EMS FSP (C11-T2) interface 10.0.0.1 255.255.255.0
EMS utility node (5149-23E)
  IP address Netmask
EMS (HOST) management interface 192.168.45.70 255.255.255.0
EMS (HOST) BMC interface 10.0.0.130 255.255.255.0
EMS management NIC management interface 192.168.45.71 255.255.255.0
EMS management NIC BMC interface 10.0.0.131 255.255.255.0

Additional notes for customer / LBS:

The root password should be set for the essserv1 SSR ID to execute commands. When prompted set the password.
Account type Account ID Password
Linux® OS root ibmesscluster

Campus connection required to be set on the P9 EMS. No default given as this unique to the customer environment. The SSR does not need to set this to finish Install Complete. The gateway may be needed to for this IP address to work properly.

   
EMS external IP C11-T3
EMS external IP gateway  
Recommended HMC2 port campus connection. Consider cabling this port to a public network and setting a campus IP. This will allow remote recovery/debug of the EMS in case of an outage. The gateway may be needed to for this IP address to work properly.
   
EMS HMC2 port IP HMC2 port
EMS HMC2 port IP gateway  
   
Utility node external IP  
Utility node external IP gateway  

The VLAN tag is required to route traffic on the BMC interface to the service network. This interface is connected to the "green" trunk ports on the switch. This is required only on ESS 3200 and ESS 3500. The default is 101.

     
ESS 3200/3500 VLAN tag VLAN TAG 101

Customer has purchased a scale license. The options are Data Access Edition (DAE) or Data Management Edition (DME). When deciding which edition to download and copy to the /serv directory you will need this information.

The Scale license field is only needed for ESS 6130 or later.

     
Scale license DAE or DME  
Note: Write down any vital information that should be shared with the customer / LBS that you encountered during execution of Install Complete.
Start of changeEnd of change
Start of changeEnd of change