 |
The Linux feature "Fixed I/O buffers" addresses Linux systems running
on z/VM 5.1 or lower. Prior to enabling fixed I/O buffers, please ensure that
you have installed the latest z/VM service level. For z/VM 5.1, you should be
at least at z/VM service level 501.
The fixed I/O buffer functionality is available for 2.4 and 2.6
kernels. Both configuration and scope of functionality differ slightly.
See your kernel specific description.
If you copy or download any software from this web site please
be aware of the Warranty Disclaimer
and Limitation of Liability.
|
To enable the fixed I/O buffers feature use the new keyword 'fixedbuffers'
on the 'dasd=...' parameter in an arbitrary place.
- Example:
dasd=7000-7002,fixedbuffers
- Note:
-
When the new feature is enabled it applies to all ECKD disks, but not to
disks that are accessed using the FBA or DIAG disciplines. The new feature
works on all DASDs with a blocksize that is equal to the pagesize.
The pagesize in today's Linux systems is 4k. The feature has no effect on
DASDs with a different blocksize. You can specify the new 'fixedbuffers'
keyword on the kernel parameter line when the DASD driver is build into
the kernel, or on the command line, when the dasd_mod module
is loaded. Typically, running your Linux on System z distribution, you will
configure the dasd parameters in /etc/zipl.conf,
run mkinitrd and run zipl. Configuring fixedbuffers
this way will ensure that the feature is permanently used with every new
IPL.
- How to verify the activation:
-
To check whether the parameter was recognized correctly you can have a
look at the boot messages in the message log. When the fixed I/O buffers
are enabled successfully you find the following line in the log:
dasd: turning on fixed buffer mode
A second way to check that the fixed I/O buffers are enabled is to view
the file /proc/slabinfo. If the feature is enabled successfully
the slabinfo file contains information for a slab cache called
'dasd_page_cache'.
|
The idea behind fixed I/O buffers is to allocate a
fixed set of pages that will act as a transfer buffer
for all read and write requests, a concept similar to
that of bounce buffers. But while bounce buffers are
meant to restrict I/O to a specific memory area, the
aim of fixed I/O buffers is to reduce the number of
different pages visible for z/VM in I/O requests (CCW
programs).
Since we want to achieve this goal while keeping maximum
flexibility at the same time, we don't actually use
a fixed set of buffer pages, instead we take those pages
from a dedicated slab cache. The advantage of the slab
cache is that it adapts itself to the workload. The
cache grows when the workload is high and it frees resources
and shrinks when the workload is low. Once its size
matches the current workload it changes in small amounts
and the set of pages used for I/O is rather stable.
Information about all slab caches is available via
proc file system. The file '/proc/slabinfo'
contains the current status. The name of the slab cache
we use for the fixed I/O buffers is 'dasd_page_cache'
so the data for that specific cache can be found by
using a combination of the instructions cat
and grep.
Example:
$ cat /proc/slabinfo | grep dasd_page_cache
dasd_page_cache 695 696 4096 695 696
1 : 60 30
Depending on the workload you are running, it may be
necessary to tune the slabcache parameters for dasd_page_cache.
Due to the limited workload measurements we have run
on kernel 2.4, we do not provide tuning rules of thumb
at this time. Please refer to the tuning recommendations
for fixed I/O buffers on kernel 2.6 for further details.
On kernel 2.4, the slabcache entries are configurable
with the limit and batchcount
parameters which are the two parameters after the colon.
Other than on kernel 2.6, there is no sharedfactor
parameter on kernel 2.4 (see Linux
kernel 2.6).
In case the system runs into low memory situations,
it may occur that memory requests of a single page can
no longer be fulfilled. In this case the kernel issues
the message '__alloc_pages: 0-order allocation
failed (gfp=0x21/0)'. The system is still
working, but it shows that it is under memory pressure.
If possible, you should increase the memory of the system.
|
|
|
To enable the fixed I/O buffers feature use the new keyword 'fixedbuffers'
on the 'dasd=...' parameter in an arbitrary place.
- Enable the fixed I/O buffer feature in SUSE SLES 9 SP1:
-
Add the option 'fixedbuffers' to the kernel command
line parameters in the file /etc/zipl.conf.
Please note that you need to add all necessary DASD devices
as well, since the DASD configuration on the command line
will be used instead of the DASD configuration done in
the past.
Example:
[SLES9FixedBuf]
target = /boot/zipl
image = /boot/image
ramdisk = /boot/initrd
parameters = "dasd=<list of all DASD devices>,fixedbuffers
root=/dev/dasda1 selinux=0 TERM=dumb elevator=cfq"
You need to run zipl (zipl -V is more informative) and reboot
your system.
- Enable the fixed I/O buffer feature in RHEL4:
-
Add the option 'fixedbuffers' in the file /etc/modprobe.conf.
Example:
options dasd_mod dasd=<list of all DASD devices>,fixedbuffers
To activate the usage of fixed buffers you need to rebuild the initrd
using the tool mkinitrd.
To update the initrd for the current running kernel:
-
cd /boot
- save the old
initrd
mkinitrd -f initrd-$(uname -r).img $(uname -r)- ensure that the name of the new
initrd matches the entry
in /etc/zipl.conf
You need to run zipl (zipl -V is more informative) and reboot
your system.
-
- Note:
-
When the new feature is enabled it applies to all ECKD and FBA disks, but
not to disks that are accessed using the DIAG discipline. You can specify
the new 'fixedbuffers' keyword on the kernel parameter
line when the DASD driver is build into the kernel, or on the command line,
when the dasd_mod module is loaded.
- How to verify the activation:
-
To check whether the parameter was recognized correctly you can have a
look at the boot messages in the message log. When the fixed I/O buffers
are enabled successfully you find the following line in the log:
dasd_devmap: turning on fixed buffer mode
A second way to check that the fixed I/O buffers are enabled is to view
the file /proc/slabinfo. If the feature is enabled successfully
the slabinfo file contains information for a slab cache called
'dasd_page_cache'.
|
The idea behind fixed I/O buffers is to allocate a fixed set of pages that
will act as a transfer buffer for all read and write requests, a concept similar
to that of bounce buffers. But while bounce buffers are meant to restrict I/O
to a specific memory area, the aim of fixed I/O buffers is to reduce the number
of different pages visible for z/VM in I/O requests (CCW programs).
Since we want to achieve this goal while keeping maximum flexibility at the
same time, we don't actually use a fixed set of buffer pages, instead we take
those pages from a dedicated slab cache. The advantage of the slab cache is
that it adapts itself to the workload. The cache grows when the workload is
high and it frees resources and shrinks when the workload is low. Once its size
matches the current workload it changes in small amounts and the set of pages
used for I/O is rather stable.
The growing and shrinking behavior of a slab cache is controlled by three parameters.
You can tune it at runtime. The current parameters of all slab caches in the
system can be accessed through the proc file system. The file '/proc/slabinfo'
contains the current setting of the tuning parameters, the 'tunables', as well
as some statistical data. The name of the slab cache we use for the fixed I/O
buffers is 'dasd_page_cache' so the data for that specific cache
can be found by using a combination of the instructions cat and
grep.
Example:
$ cat /proc/slabinfo | grep dasd_page_cache
dasd_page_cache 0 0 4096 1 1 : tunables 24 12 8 : slabdata 0
0 0
The three tuning parameters (called tunables) are limit, batchcount,
and sharedfactor. To set the tunables, you can use the same proc
interface that is used to read these values. To set the tuning parameters for
a given slab cache, write it's name and the new values for limit, batchcount,
and sharedfactor to it:
echo "<name> <limit> <batchcount> <sharedfactor>"
> /proc/slabinfo
There is no simple rule to determine the optimal settings for the tuning parameters,
since these settings depend on the actual workload. The default settings are
appropriate for workloads with rather random I/O, while sequential I/O needs
larger values. To find best settings, start with the default settings, measure
the performance under the relevant workload and then increase the numbers iteratively.
A few rules of thumb:
- Limit should be about two to four times higher than batchcount.
- Increase batchcount in steps of 25.
- Keep sharedfactor at 8.
Example:
- Start with the default values (24, 12 and 8), since they are often sufficient.
- Measure performance, if performance is not sufficient then increase batchcount
to 25 and limit to 75
echo "dasd_page_cache 75 25 8" > /proc/slabinfo
- If the results of the performance measurement are still not sufficient go
on with:
echo "dasd_page_cache 150 50 8" > /proc/slabinfo
- If a third iteration is necessary:
echo "dasd_page_cache 225 75 8" > /proc/slabinfo
- Go on with these iterations until the performance measurement shows the
desired result.
|
|
|
|
|
|
