For both SCSI and ECKD devices, there are possibilities included
in Linux on System z to collect various statistical data such as
I/O request data sizes or I/O latencies.
Statistical data on FCP devices can be collected
starting with SUSE Linux Enterprise Server 9 SP3 + maintenance
(kernel version 2.6.5-7.283 and higher)
with SUSE Linux Enterprise Server 10 GA (kernel version 2.6.16.21-0.8
and higher)
For gathering statistics data, the following steps are necessary:
Prerequisites:
To collect performance data on adapters and logical unit
numbers (LUNs), ensure that the parameter CONFIG_STATISTICS
in the kernel config file is set to yes, which is the default:
CONFIG_STATISTICS=y
Depending on your distribution the files for zfcp
statistics can be found as follows:
For SLES10 and later, depending on where debugfs
is mounted: <mount_point_debugfs>/statistics.
For example, if debugfs is mounted at directory
/sys/kernel/debug/, all the collected statistics
data can be found at /sys/kernel/debug/statistics/.
For SLES9, depending on where proc is
mounted (SLES9 does not use debugfs): <mount_point_proc>/statistics.
For each device – adapter as well as LUN – a subdirectory
is created when mounting the device. A subdirectory is named
zfcp-<device-bus-id> for an adapter and
zfcp-<device-bus-id>-<WWPN>-<LUN>
for a LUN.
Each subdirectory contains two files, a data file and
a definition file.
To switch on data gathering for the devices, enter
echo on=1 > definition
To switch off data gathering for the devices, enter
To process a comparison of the SCSI and ECKD I/O request
data sizes, the different data representations have to be
converted.
In the FCP statistics, all the occurring I/O request data
sizes are listed in KB in hexadecimal format. ECKD displays
its statistics results in KB categories of powers of two.
We converted the FCP I/O request data sizes to decimal
numbers and grouped the FCP values into the corresponding
ECKD categories to make them comparable.
The way SCSI and ECKD merge I/O requests is different. The
velocity of processing considerably impacts the merging behavior.
SCSI sends out all requests to the storage server at once
until the memory capacity of the adapter card is exhausted.
Merging in the Linux block device layer is hence not done
as broadly as it is for ECKD.
In the ECKD case, requests remain queued up and are processed
only one at a time with the next request having to wait
until the previous one has finished processing.
There are three types of I/O latencies being reported:
Channel latency
The channel latency roughly corresponds to the time a
request spent in the channel.
Fabric latency
The fabric latency is the time a request spent outside
the System z machine. This includes latencies caused by
the SAN and the SCSI device (storage server).
From these values you can calculate the passthrough latency:
Passthrough latency
The passthrough latency is the delay caused by QDIO (the
FCP transport between Linux device driver and FCP channel
adapter) and, if applicable, a hypervisor which makes FCP
subchannels available to a hosted Linux system. The passthrough
latency can be estimated as
where the overall latency appears with the prefix "latencies_scsi_"
in the output.
The overall latencies are given in milliseconds. Fabric and
channel latencies are given in nanoseconds. To show them in
the same charts, we converted fabric latencies to milliseconds
as well. The channel latencies are very small numbers and
were thus neglected for our considerations.
