FCP statistics (SLES9 / SLES10)

This illustration shows the starting and end points of different kinds of statistics.

Back to top

For gathering statistics data, the following steps are necessary:

Prerequisites:

To collect performance data on adapters and logical unit numbers (LUNs), ensure that the parameter CONFIG_STATISTICS in the kernel config file is set to yes, which is the default:

CONFIG_STATISTICS=y

1. Depending on your distribution the files for zfcp statistics can be found as follows:
   
   • For SLES10 and later, depending on where debugfs is mounted: <mount_point_debugfs>/statistics.
     For example, if debugfs is mounted at directory /sys/kernel/debug/, all the collected statistics data can be found at /sys/kernel/debug/statistics/.
   • For SLES9, depending on where proc is mounted (SLES9 does not use debugfs): <mount_point_proc>/statistics.
   For each device – adapter as well as LUN – a subdirectory is created when mounting the device. A subdirectory is named
   
   • zfcp-<device-bus-id> for an adapter and
   • zfcp-<device-bus-id>-<WWPN>-<LUN> for a LUN.

2. Each subdirectory contains two files, a data file and a definition file.

3. To switch on data gathering for the devices, enter

   echo on=1 > definition
   

   To switch off data gathering for the devices, enter

   echo on=0 > definition
   

   By default, data gathering is turned off.

   To reset the collected data to 0, enter

   echo data=reset > definition
   

Back to top

To process a comparison of the SCSI and ECKD I/O request data sizes, the different data representations have to be converted.

In the FCP statistics, all the occurring I/O request data sizes are listed in KB in hexadecimal format. ECKD displays its statistics results in KB categories of powers of two.

We converted the FCP I/O request data sizes to decimal numbers and grouped the FCP values into the corresponding ECKD categories to make them comparable.

The way SCSI and ECKD merge I/O requests is different. The velocity of processing considerably impacts the merging behavior.

• SCSI sends out all requests to the storage server at once until the memory capacity of the adapter card is exhausted. Merging in the Linux block device layer is hence not done as broadly as it is for ECKD.
• In the ECKD case, requests remain queued up and are processed only one at a time with the next request having to wait until the previous one has finished processing.

Back to top

There are three types of I/O latencies being reported:

Channel latency

The channel latency roughly corresponds to the time a request spent in the channel.

 

Fabric latency

The fabric latency is the time a request spent outside the System z machine. This includes latencies caused by the SAN and the SCSI device (storage server).

From these values you can calculate the passthrough latency:

Passthrough latency

The passthrough latency is the delay caused by QDIO (the FCP transport between Linux device driver and FCP channel adapter) and, if applicable, a hypervisor which makes FCP subchannels available to a hosted Linux system. The passthrough latency can be estimated as

passthrough latency = overall latency – (channel latency + fabric latency)

where the overall latency appears with the prefix "latencies_scsi_" in the output.

The overall latencies are given in milliseconds. Fabric and channel latencies are given in nanoseconds. To show them in the same charts, we converted fabric latencies to milliseconds as well. The channel latencies are very small numbers and were thus neglected for our considerations.

Back to top

'cat /sys/kernel/debug/statistics/zfcp-0.0.1700-0x5005076303010482-0x4014400500000000/data':

request_sizes_scsi_write 0x10000 8976
request_sizes_scsi_write 0x20000 321
request_sizes_scsi_write 0x1000 314
request_sizes_scsi_write 0x4000 310
request_sizes_scsi_write 0xc000 255
request_sizes_scsi_write 0x2000 197
...
request_sizes_scsi_read 0x10000 10840
request_sizes_scsi_read 0x1000 3581
request_sizes_scsi_read 0x4000 168
...
request_sizes_scsi_nodata 0
request_sizes_scsi_nomem 0
request_sizes_timedout_write 0
request_sizes_timedout_read 0
request_sizes_timedout_nodata 0
latencies_scsi_write <=0 0
latencies_scsi_write <=1 595
latencies_scsi_write <=2 1071
latencies_scsi_write <=4 1869
latencies_scsi_write <=8 3122
latencies_scsi_write <=16 4888
latencies_scsi_write <=32 309
latencies_scsi_write <=64 12
latencies_scsi_write <=128 45
latencies_scsi_write <=256 108
latencies_scsi_write <=512 35
latencies_scsi_write <=1024 5
latencies_scsi_write >1024 1
latencies_scsi_read <=0 0
latencies_scsi_read <=1 14083
latencies_scsi_read <=2 150
...
latencies_scsi_nodata <=1 0
latencies_scsi_nodata <=2 0
latencies_scsi_nodata <=4 0
...
pending_scsi_write 12060 1 23.275 32
pending_scsi_read 14768 1 1.010 2
occurrence_erp 0
occurrence_eh_reset 0

channel_latency_write <=0 0
channel_latency_write <=1000 0
channel_latency_write <=2000 0
channel_latency_write <=4000 0
channel_latency_write <=8000 0
channel_latency_write <=16000 0
channel_latency_write <=32000 34
channel_latency_write <=64000 451
channel_latency_write <=128000 814
channel_latency_write <=256000 7766
channel_latency_write <=512000 1183
channel_latency_write <=1024000 768
channel_latency_write <=2048000 738
channel_latency_write <=4096000 302
channel_latency_write <=8192000 4
channel_latency_write <=16384000 0
channel_latency_write >16384000 0
channel_latency_read <=0 0
channel_latency_read <=1000 0
channel_latency_read <=2000 0
...
channel_latency_nodata <=0 0
channel_latency_nodata <=1000 0
channel_latency_nodata <=2000 0
...
fabric_latency_write <=0 0
fabric_latency_write <=1000000 981
fabric_latency_write <=2000000 1111
...
fabric_latency_read <=0 0
fabric_latency_read <=1000000 14144
fabric_latency_read <=2000000 144
...
fabric_latency_nodata <=0 0
fabric_latency_nodata <=1000000 0
fabric_latency_nodata <=2000000 0

Back to top