Shared Ethernet Adapter (SEA) Failover with Load Balancing
nagger 100000MRSJ Comments (10) Visits (30195)
Update: The developers and the manuals call this Load Sharing but most people think it is called Load Balancing. Perhaps, balancing gives the wrong impression of fine grain packet by packet balancing where we actually have higher level, cruder splitting of the work with Sharing. Below I use the word Balancing but mean Sharing.
I have got a few questions recently on how to set this up as there are announcement with near zero information on setup, the configuration needed and a worked example. So here goes.
For a long time now we have had SEA Failover where the VIOS pair work together to provide a simple to set up at the client VM (LPAR) redundant path to the network. A single virtual Ethernet network is managed between two Virtual I/O Servers (VIOS). The one with the higher priority (lower number) is the primary and does all the network bridging I/O and the secondary does nothing unless the primary is taken down or fails. Then the secondary takes over and does all the network bridging until the primary returned when the primary takes over again.
Here is my diagram of a non-trivial network setup with one SEA connected to many internal virtual networks:
Note: All the network I/O is going via VIOS a. This also assuming the other ent* adapters are present but not involved in the example.
To create the SEA on VIOS a you rung the following command
mkvdev -sea ent0 -vadapter ent3,ent4,ent5, ent6 -default ent3 -defaultid=10 -attr ha_mode=auto ctl_chan=ent9
In this symmetric VIOS example it is the same command for VIOS b. The all important Priority which decides which is Primary and Secondary VIOS is set at the VIOS Virtual Ethernet Adapter Properties panels on the HMC as below:
VIOS b will of course have a higher Priority number meaning it is the Secondary VIOS.
Now let us move on to SEA Failover with Load Balancing
This is all wonderful and simple to setup up but it does mean only half of the physical network adapters are used at one time. If you are running 10 Gbit network adapters, which are fairly expensive, that seems like wasting resources and hence the new Load Balancing feature to use of all the network adapters for higher bandwidth and better latency (when busy). In the above, diagram when both VIO Servers are running ALL the packets will exit the machine via VIOS a and nothing via VIOS b.
Now some people may have the unrealistic idea that with SEA Failover with Load Sharing will get you to network heaven!! Like every other packet from the client VM would go to different VIOS and its SEA. This is impossible as the client VM only "sees" the one virtual Ethernet and can't control the destination VIOS and it would be impossible for the VIO Server to some how co-operate to acheive a perfect spilt in traffic and organising that would slow the network down enormously. If that level of concurrency is required you need to set-up multiple virtual Ethernets on the client VM and use LInk-Aggregation (also called Ether-Channel or Teaming) at the client VM. The down side is a more complicated set-up on every single client VM.
The SEA Failover with Load Balancing pair of VIOS have to vigorously agree which VIOS does what and this is done at the Trunk Adapter level. This is achieved with the Secondary VIOS communicating with the Primary VIOS to volunteer to take over a set of Trunk Adapters (now called Ethernet bridging on the HMC). Once acknowledged they split the network I/O work between them. If later you take a VIOS down or there is a genuine failure the surviving VIOS takes on all the network I/O .... of course.
With both VIOS running we have a configuration like this:
Note: The network I/O is going via both VIO Servers.
If you have a very simple network configuration like one internal virtual network and a SEA bridging to the outside physical Ethernet then the traffic can't be split between VIO Servers , obviously. This is what we run a lot of the time in my Lab environment as our needs are simple. But in more production like environments there is typically many virtual networks for user access, systems administration, multiple tier networks and a separate backup network.
If you already have a suitable SEA running you can change its mode with (assuming you SEA device is ent10) - this can be dynamically changed:
chdev -dev ent10 -attr ha_mode=sharing
Here are some notes from the developers that might help out:
entstat -d ent10 | grep -i state
I hope this helps you get Shared Ethernet Adapter Failover with Load Sharing working.
Thanks, Nigel Griffiths