Turbo charge inter-LPAR communications using super jumbo frames
AnthonyEnglish 270000RKFN Comments (18) Visits (23039)
There are a few strategies for speeding up communications between LPARs on the same physical server. One of those is using virtual adapters.
Those virtual adapters can run at high speeds. You can turn on this turbo-charge feature by setting the MTU size to 65280 Bytes, which are actually SUPER jumbo frames, according to Wikipedia. Whatever that means, it sounds good, doesn't it?
"Great! Now prove it."
All very well, but how do you test it? I knew you'd ask.
The document Basic Monitoring of I/O on AIX explains that ftp can be used to perform a memory to memory copy between two LPARs. As that document explains:
[ ftp ] is a great tool to analyze issues on the network connectivity since this excludes side effects of not network related I/O due to slow disks, CPU, etc."OK. Less talk, more action."
I get the message.
I'm copying from one LPAR to another on the same managed system. I have a virtual adapter on each LPAR and they each have IP addresses in a private subnet.. On each LPAR, the virtual adapter happens to be en1. The MTU size is currently set to 1500, as you can see from the lsattr command:
mtu 1500 Maximum IP Packet Size for This Device TrueI'm going to do two tests:
Test 1: virtual adapters, MTU 1500
Test 1 results: 170 seconds.
Test 2: virtual adapters, MTU of 65280
Now I'll ramp up the MTU size on the virtual adapters on each LPAR using the chdev command. There's no outage or downtime involved.
# chdev -l en1 -a mtu=65280And now for the test, which will hopefully beat the record so far of 170 seconds:
Using the higher MTU on the virtual adapter brought it down from 170 seconds to 45 seconds. In other words, on virtual ethernet the MTU of 65280
Of course that's not a serious load test, but I did run some other tests and throughput speeds were consistently better using the higher MTU size.
What this means for you
Here are three instances where this simple change might be handy. If you have two LPARs on the same physical server you could communicate between them using their virtual ethernet adapters for:
Maybe it's time to think about turbo charging your comms between LPARs.