Learn Linux, 302 (Mixed environments): Performance tuning

Measure and improve your Samba performance

In preparation for taking the Linux Professional Institute Certification exam LPI-302 for systems administrators, learn how to measure Samba performance and make Samba work faster.

Sean A. Walberg, Senior Network Engineer

Photo of Sean WalbergSean Walberg is a network engineer and the author of two books on networking. He has worked in several industries, including health care and media.



20 December 2011

Also available in Chinese Russian Japanese Portuguese

About this series

This series of articles helps you learn Linux systems administration tasks. You can also use the material in these articles to prepare for the Linux Professional Institute Certification level 3 (LPIC-3) exams.

See our developerWorks roadmap for LPIC-3 for a description of and link to each article in this series. The roadmap is in progress and reflects the current objectives (November 2010) for the LPIC-3 exams. As each article is completed, it is added to the roadmap.

In this article, learn how to:

  • Measure Samba performance
  • Optimize Samba memory usage
  • Improve file-transfer speeds in a Server Message Block/Common Internet File System environment

This article helps you prepare for Objective 315.3 in Topic 315 of the Linux Professional Institute's (LPI) Mixed Environment Specialty exam (302). The objective has a weight of 1.

Prerequisites

To get the most from the articles in this series, you should have an advanced knowledge of Linux and a working Linux system on which you can practice the commands covered in this article. You should also have a good understanding of TCP/IP networking.


Measuring Samba performance

About the elective LPI-302 exam

Linux Professional Institute Certification (LPIC) is like many other certifications in that different levels are offered, with each level requiring more knowledge and experience than the previous one. The LPI-302 exam is an elective specialty exam in the third level of the LPIC hierarchy and requires an advanced level of Linux systems administration knowledge.

To get your LPIC-3 certification, you must pass the two first-level exams (101 and 102), the two second-level exams (201 and 202), and the LPIC-3 core exam (301). After you have achieved this level, you can take the elective specialty exams, such as LPI-302.

Before you can improve something, you must be able to reliably measure it. Measure something, make changes, measure again, and compare results. In the case of Samba, you need to measure:

  • Response time and throughput of a client under idle server conditions
  • Response time and throughput of a client under a given server load
  • Server characteristics under a given load
  • Maximum server capacity in terms of clients or throughput

Measuring response times gives you an idea of what a typical client would expect under the test conditions. For this to work, your test cases should approximate actual client load. For example, seeing how fast you can repeatedly request the same file is not the same as copying a directory containing a mix of small and large files.

Measuring server characteristics under a given load gives you some idea of how much room you have to grow. If, under load, your server is struggling to keep up, then you know you don't have much capacity beyond the test load. The same techniques are used to measure a production server's load and extrapolate capacity.

Finally, the so-called "torture tests," where you throw all you can at a server and see where it breaks, provide interesting information but are not as useful as the other types of tests. If the goal is to prove that your server can handle a certain load, then this type of test will do. Such tests are often more helpful for measuring the lower-level characteristics, such as the maximum disk I/O capacity of a server.

Samba is a network server, so it is important to reasonably simulate the network being used. If your clients are 50 milliseconds away from the server, the effects of network latency will be more pronounced than with local clients. This information directs your tuning priorities accordingly.

Designing a test

You could buy a fairly expensive device that simulates client traffic and takes precise performance measurements. And if you're publishing benchmarks or developing the server hardware, such a device might be a good option. However, if you're interested in tuning your own server, and are faced with the usual constraints of time and money, you're probably not looking for an expensive tool that you'll have to learn how to use.

As an example, the first test assesses random read performance by downloading a large number of files with the Samba command-line client. The primary concern is, "how fast can this directory be downloaded?"

Like any well-behaved UNIX® utility, the smbclient tool can read its list of instructions from the standard input. The following snippet shows a series of commands for downloading the contents of a directory:

prompt
recurse
mget smbtest

The prompt command suppresses the client from asking you for confirmation of downloads, and recurse indicates that you want to descend into directories when you download multiple files. Finally, mget smbtest instructs the client to start downloading the smbtest directory. Fill that directory with a few hundred megabytes of test files, and you have all you need to test performance.

To run the test, connect to your share with smbclient, and redirect standard input to your file. Listing 1 shows how you do it.

Listing 1. Running the test
$ time smbclient '\\192.168.1.1\test' password < instructions 
Domain=[BOB] OS=[Unix] Server=[Samba 3.5.8-75.fc13]
getting file \smbtest\file2 of size 524288000 as file2 (5323.2 kb/s) (average 5323.2 kb/s)
getting file \smbtest\file1 of size 139460608 as file1 (5275.3 kb/s) (average 5313.0 kb/s)

real	2m2.289s
user	0m0.509s
sys	0m4.580s

The command in Listing 1 starts with the time command, which times how long the command specified in the rest of the arguments takes to run. The command itself is smbclient, and the arguments to that are the name of the share and the user's password. You can add the typical arguments, such as -U to pass an alternate user name if your environment requires it. Finally, < instructions redirects the standard input of smbclient to the file called instructions, which contains the instructions from the first code snippet. The result of this command is a timed batch copy of several files.

Build your own feed

You can build a custom RSS, Atom, or HTML feed so you will be notified as we add new articles or update content. Go to developerWorks RSS feeds. Select Linux for the zone and Articles for the type, and type Linux Professional Institute for the keywords. Then, choose your preferred feed type.

The result of the command is a list of the files transferred, along with an average transfer rate per file. The output of time is added at the end, showing that copying roughly 664MB of files took 2 minutes and 2.289 wall-clock seconds. This is now the benchmark. If you make any changes and the test takes longer than 2 minutes, 2 seconds, then you're making things worse with your changes.

If you want to test just Samba parameters and negate the effects of local disk and caching, you can run the test several times and take the last measurement. Doing so ensures that the operating system caches as much as possible and minimizes disk access. As you test your changes, make sure you have a similar amount of free memory on your server; otherwise, the differences in the amount of cached data might alter the results of your experiment.

Viewing Samba's status

Looking at a server's CPU, memory, disk, and network information gives you some good information about the health of the server itself but provides no context for what the application is doing. Samba comes with a helpful utility called smbstatus that shows current connections and file activity; it's also good for both performance tuning and troubleshooting.

Listing 2 shows the smbstatus command in action.

Listing 2. The smbstatus command
$ smbstatus
lp_load_ex: refreshing parameters
Initialising global parameters
params.c:pm_process() - Processing configuration file "/etc/samba/smb.conf"
Processing section "[global]"
Processing section "[homes]"
Processing section "[printers]"
Processing section "[extdrive]"

Samba version 3.5.8-75.fc13
PID     Username      Group         Machine                        
-------------------------------------------------------------------
17456     fred         fred       macbookpro-d0cd (::ffff:192.168.1.167)

Service      pid     machine       Connected at
-------------------------------------------------------
fred         17456   macbookpro-d0cd  Mon Jul 18 07:36:46 2011
extdrive     17456   macbookpro-d0cd  Mon Jul 18 07:36:46 2011

Locked files:
Pid    Uid   DenyMode   Access      R/W        Oplock      SharePath   Name   Time
----------------------------------------------------------------------------------
17456  505   DENY_NONE  0x100081    RDONLY     NONE        /home/fred   .  
17456  505   DENY_NONE  0x100081    RDONLY     NONE        /home/fred   Documents

Listing 2 shows the current activity of the Samba server. One user, fred, is connected, and he has two shares mounted (fred and extdrive). There are also two locked directories.


Network tuning

Samba is a daemon that primarily sends and receives packets over the network. Several parameters alter how the packets are sent and how Samba interacts with the underlying operating system; these parameters can have a drastic effect on performance.

The basic options

Many daemons provide the best possible level of service to everyone and do not discriminate among clients. If the service is overloaded, everyone gets the same bad service. Contrast this to a telephone network: if congestion occurs, people aren't allowed to make new calls, but existing calls continue as though nothing's wrong. If you artificially limit certain options in Samba, you can prevent getting to a resource starvation situation.

The max connections parameter limits the number of connections you can have to a particular Samba server. Each connection takes memory and CPU resources, so it's possible to overload a server by having too many connections. By default, unlimited connections are allowed. If you need to limit a busy server, you can specify a hard limit with max connections.

The max smbd processes parameter controls the maximum number of processes that can be run. This parameter is similar to max connections but controls the number of processes resulting from the connections.

The more you log with log level, the more server resources are spent writing logs to disk. Keeping this value at 1 or 2 limits the amount of logs written to disk and saves more resources for serving clients.

Setting socket options

When an application asks the operating system to open a network connection, the application can also ask that the operating system treat the packets a certain way using socket options. Socket options can enable or disable network performance tweaks, set particular quality-of-service bits on the packets, or set kernel-level options on the socket.

The socket options command can control the type of service (TOS) bits on the packets. The TOS bits tell routers along the way how the traffic should be treated. If the routers are configured to respect these bits, the traffic can be handled in the manner the application asked for. The IPTOS_LOWDELAY keyword is most appropriate for low-delay networks such as LANs, while IPTOS_THROUGHPUT is for higher-latency WAN links. Your network may be configured differently, so it's possible that using the options might have the opposite effect.

The TCP_NODELAY option disables the Nagle algorithm (see Resources for a link to more information), which is more appropriate for chatty protocols, at a cost of more packets being sent.

If you have firewalls or other devices that keep state in your network, you may be interested in the SO_KEEPALIVE option, which turns on TCP keepalives. These periodic packets keep the connection open and keep the state fresh inside firewalls. Otherwise, the firewall will drop the packets, and it will take some time for the client to realize it has to reconnect to the server.


Beyond tuning

Although playing with different settings and trying to come up with optimal configurations may be fun, some items outside the Samba configuration can really hamper performance. Any time the server is doing something other than reading data from disk and sending it to a client or receiving incoming network traffic and writing it to disk, you're making the operation take longer.

Ethernet errors

If a packet is lost in transmission, the kernel has to notice that the packet has gone missing and request a retransmission. This process can slow down a fast conversation, especially if the two sides are separated by high latency. One common source of packet loss is mismatched settings between the switch and the server. Either auto-negotiation fails to come up with a correct match or one side is set statically and the other side is still auto-negotiating. This discrepancy results in errors on both sides. You can look for such errors with the netstat command:

# netstat -d -i 2
Kernel Interface table
Iface       MTU Met    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth1       1500   0  5258404      0      0      0  3024340      0      0      0 BMRU
eth1       1500   0  5258409      0      0      0  3024341      0      0      0 BMRU
eth1       1500   0  5258411      0      0      0  3024342      0      0      0 BMRU

This code shows the multipurpose netstat command looking for interface errors every 2 seconds with the -d -i 2 parameters. Every 2 seconds, the status of the available interfaces is shown. In the example above, the RX-OK and TX-OK columns show that packets are flowing in and out. The errors, drops, and overruns are all 0 in both directions, showing that there has been no packet loss.

If you see errors, determine what speed/duplex is being used with mii-tool or mii-diag. Listing 3 shows how to check your network settings.

Listing 3. Verifying network settings
# mii-tool 
eth1: negotiated 100baseTx-FD, link ok
# mii-diag eth1
Basic registers of MII PHY #24:  3000 782d 0040 6177 05e1 41e1 0003 0000.
 The autonegotiated capability is 01e0.
The autonegotiated media type is 100baseTx-FD.
 Basic mode control register 0x3000: Auto-negotiation enabled.
 You have link beat, and everything is working OK.
 Your link partner advertised 41e1: 100baseTx-FD 100baseTx 10baseT-FD 10baseT.
   End of basic transceiver information.

Listing 3 starts with the mii-tool command to give a short summary of the active interfaces. The results show that the interface has negotiated at 100/Full. mii-diag shows more detailed information and might be more suitable for sending to your network team if you have to escalate the issue. The mii-tool command can be used to fix the speed and duplex, though in practice it's better to have the link auto-negotiated.

Maintaining TDB files

In "Learn Linux, 302 (Mixed environments): Trivial Database files," you learned about the trivial database files (TDBs) that Samba uses to keep state. If something is wrong with these databases, your server will have to perform more work, such as unnecessary disk seeks, or fail to cache information from remote services. Fortunately, you can check the TDB files for corruption and fix them if there's a problem. You can delete the temporary TDB files after shutting down Samba, and they'll be re-created on startup. For others, be sure to perform a backup, and then verify the files with tdbbackup -v.

Look at the client

Sometimes, the client may be the cause of slow performance. The client might have duplex problems, it might have malware, or some other problem may be occurring. Using a consistent client for your testing can help you eliminate the server as a potential source of slowness.


Conclusion

This was the final article in the LPIC exam 302 series. Review your notes, and take the exam! Good luck!

Resources

Learn

Get products and technologies

Discuss

  • Get involved in the developerWorks community. Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Select information in your profile (name, country/region, and company) is displayed to the public and will accompany any content you post. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Linux on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Linux
ArticleID=781961
ArticleTitle=Learn Linux, 302 (Mixed environments): Performance tuning
publish-date=12202011