Maximum Virtual Slots used up on your VIO server? Maybe not...
brian_s 270002K5X3 Comments (202) Visits (13195)
Over my career as an AIX administrator I have run in to problems with the "Maximum Virtual Adapters" LPAR profile setting many times. This setting controls the highest virtual slot number that you can use on an VIO or AIX LPAR. It is not dynamically changeable and requires a reboot to modify.
By default when you create a VIO server the Maximum Virtual Adapters is set to only 20. Any time you are creating a VIO server I recommend setting this setting much, much higher. The higher you set this to the more memory overhead, so don't go crazy. Determine how many slots it will take to support the number of LPAR's you plan on having on the system, and then at least double that number to come up with what you want to set this to. Also @nixysug pointed out that if you have the maximum virtual adapters set to above 1,000 you might have issues with live partition mobility (for more info see http
Lets suppose you have a dual VIO system that was setup with the default Maximum adapters set to 20. You are using either Virtual SCSI or Virtual Fibre Channel (doesn't really matter which one as the slots work basically the same way). You have come up with a slot numbering convention where slots 10 and under are used for Virtual Ethernet. For the VSCSI/VFC slots, your convention is to start at slot 11 and have 2 slots per AIX LPAR. One slot will go to VIO1 and the other slot to VIO2. You decide to have the odd number slots go to VIO1 and the even number slots go to VIO2. This is a very common slot numbering convention and is often recommended/used in IBM documentation like Redbooks.
The system will not allow you to create any slots equal to or greater than whatever the maximum virtual adapters setting is set to. So if you have a maximum virtual adapters to the default, 20, then you will only be able to use slots numbered 19 or below.
As previously mentioned the only way to change this maximum virtual adapters setting is to change the VIO server profile, shut the VIO server all the way down, and then re-activate it. Depending on your environment having a downtime on the VIO server might be a big deal, even if you have dual VIO servers.
If you have a urgent request to build a new LPAR, here are your options at this point:
I have found that there is a lot of confusion out there about virtual slot numbers on POWER servers. For the first several years I worked on these servers I had a lot of incorrect assumptions about how the slots worked.
A lot of people are under the incorrect impression that the client and server adapter numbers need to match. For example, some people think that if you create an adapter with slot 11 on the LPAR that the slot on the VIO server also needs to be slot 11. This is totally incorrect, and you can in fact have VSCSI client slot 11 connect to VSCSI VIO server slot 16 with no problems at all.
Another common misconception is that the slot numbers are "global" across the entire system and must be unique across the entire system. In fact, it is possible to setup multiple LPAR's that all use the same virtual slot number. For example, you could have 100 LPAR's and have every one of them use virtual slot numbers "4" and "5" for their virtual SCSI client adapters. Here is an example of a slot numbering convention where this is done (notice each LPAR uses slot 4 and 5 for the client adapters):
Let's suppose we want to build 2 additional LPAR's and not have a VIO downtime.
One method we could use to build these 2 new LPAR's right away and avoid a VIO downtime is to break our original slot numbering convention. For the other LPAR's we had used slots "11" and "13" on VIO1 and slots "12" and "14" on VIO2. Because these slot numbers do not need to be globally unique across the system there is nothing from stopping us from using slots "11" and "13" on VIO2 and slots "12" and "14" on VIO1 (flip-flopped from how the other LPAR's had been setup).
So the solution is to build the 2 new LPAR's like this:
p6_aix5 client slot 11 mapped to VIO2 slot 11.
p6_aix5 client slot 12 mapped to VIO1 slot 12.
p6_aix6 client slot 13 mapped to VIO2 slot 13.
p6_aix6 client slot 14 mapped to VIO1 slot 14.
Below is a diagram of how this setup would look. Note that slots 11 are used by p6_aix5 and p6vio2. Slots 11 are also used by p6_aix1 and p6vio1; but as discussed this isn't a problem.
A tool that might help you out with understanding and validating your Virtual Slot configuration is the "pslot" tool. This is a Perl program I wrote that will visualize and validate your virtual slots. It was used to create the diagrams in this article. It is free and open source, and can be downloaded from http