We have been considering reimplementing our GPFS storage system (built using x3650's SAS-attached to DCS3700's) to see if we can gain some better performance by changing the fs block size, using larger inodes, etc. To plan, we have been testing I/O performance to a DCS3700 at the block device level, so we know that while accessing data on an 8+PQ LUN performs well when accessing full stripes on stripe boundaries, accessing smaller units of storage is, well, dismal.
We know it is recommended that the stripe size match the file system block size, but then there is this notion of the "subblock". Will GPFS ever directly access the raw block device using subblock-sized I/Os? Or are subblocks only an allocation unit within the file system, and GPFS only does full block sized I/O to the block device?
I know I can use several stripes of the LUN to make up a file system block without losing much performance, but no where near 32!