NOW FOR THE I/O

I highlighted some of the excellent AIX performance tuning resources here. They particularly dealt with memory and paging space. Now to I/O tuning.

Fewer larger LUNs and checking queue_depth

Jaqui Lynch has the follow up article in her two-part performance tuning series. This one's on Disk I/O and Network Tuning. She deals especially with queue_depth and explains why one great big LUN with lots of spindles may need to have this tuned:

The trend in the industry right now is to provide fewer, larger hdisks to the server. For example, the server may be given one 500 GB hdisk that’s spread across several disks in the disk subsystem, rather than being given 10, 50 GB or five 100 GB hdisks. However, I/O performance depends on bandwidth, not size. While that data may be spread across multiple disks in the back end, this doesn’t help with queuing in the front end. At the server, the hdisk driver has an in-process and a wait queue. Once an I/O is built in the JFS2 buffer, it gets queued to the LUN (hdisk). Queue_depth for an hdisk (LUN) represents the number of in-flight I/Os that can be outstanding for an hdisk at any given time.

Jaqui also covers monitoring fibre channel adapters with the fcstat command. The changes will need to be done with some caution, but as I/O is more commonly the bottleneck on a slow system, it's worth looking into the systems you work on, especially the ones that are so slow that people have given up complaining about them and started complaining about you.

Stunning improvement

I implemented the memory tuning parameters on a SAP production system and it has had a marked improvement on paging space (from 24% to not in use at all now), and the dreaded revolutions of the clock hand - a complete scan of memory looking desperately for something spare - is down to 0. The vmstat -s is looking much healthier, thanks to Jaqui Lynch's first article, Paging, Memory and I/O Delays.

LVM Monitoring

There is also Part II of the developerWorks article on Optimizing AIX 7 performance - Monitoring Logical volumes. This is especially helpful for placement of logical volumes if you're using local disk.

There are also good examples of how to monitor a volume group using the lvmstat command. You need to enable statistics first using

lvmstat -v vgname -e

You can do this online - no need to varyoff and varyon the volume group.

Here's an example of the LVs in a volume group on a system I've been looking at. Good argument for naming your logical volumes according to the mount points rather than using lv00, lv01 etc.









When you've got a performance problem, sometimes the solution is to throw more iron at, but a simple tweak to a parameter here and there can have a vast improvement and also save money for more important things for the company ... like paying you.


Tags:  fcstat tuning vmstat iostat memory fibre queue_depth adapter disk sat performance i/o channel aix lvmstat