Overview

In this article, learn about these concepts:

• Backing up Samba Trivial Database (TDB) files
• Restoring TDB files
• Identifying TDB file corruption
• Editing/listing TDB file content

This article helps you prepare for Objective 310.3 in Topic 310 of the LPI's Mixed Environment specialty exam (302). The objective has a weight of 1.

Prerequisites

To get the most from the articles in this series, you should have an advanced knowledge of Linux and a working Linux system on which you can practice the commands covered in this article. In particular, this article assumes that you have a working knowledge of Linux command-line functions and at least a general understanding of the purpose of Samba as covered in "Learn Linux, 302 (Mixed environments): Concepts". To perform the actions described in this article, you must have the Samba software installed. Some actions require that you have a working Server Message Block (SMB)/Common Internet File System (CIFS) network at your disposal.

Back to top

Understanding TDB files

Samba stores a lot of information at run time, from local passwords to a list of clients it is expecting to hear from. Some of this data is short lived and can be discarded when Samba is restarted, but some of it is permanent and should not be lost. The data can either be too large or infrequently accessed to keep in memory, or it should persist across restarts. To address these requirements, the Samba team created the Trivial Database. It is effectively a key-value store, meaning that data is stored and retrieved by means of a unique key and there is no joining of tables as in a relational database. Key-value stores—and TDB in particular—are designed to be fast ways to store data to disk and get it back.

Key-value stores

There are many alternatives to TDB, such as the GNU Database Manager (GDBM), but the Samba project had a special need for multiple processes writing to the database at the same time as well as support for locking internal pieces of data. Therefore, the team built their own database manager and called it the Trivial Database Manager. TDB has been extended to support clustered operation through the Clustered TDB (CTDB) project and is available for use by other projects.

If you are familiar with relational databases such as IBM DB2, MySQL, or PostgreSQL, then you will find key-value stores primitive in comparison. Where a relational database may have a table with several columns, a key-value store effectively has two columns: a key column and a value column. Relational databases use Structured Query Language (SQL) to pull information from multiple tables and columns, but key-value stores are limited to operations on the key column.

Because of their simplicity, key-value stores are useful for a smaller set of jobs but can do those jobs very quickly. A key-value store is basically a hash table on disk, so the location of a piece of data can be guessed without having to consult an index.

The TDB in particular was written to handle many concurrent writers and binary data such as internal data structures. Therefore, the fastest way to store and retrieve this data is in binary format. Binary storage means that the database file can be smaller, and there is no requirement to translate data between different formats as data moves in and out of the database.

TDB files used in Samba

Samba stores its TDB files in a few different places. The easiest way to find these files is to look at the output of smbd -b, which is shown in Listing 1.

	
# smbd -b
...
Paths:
   SBINDIR: /usr/sbin
   BINDIR: /usr/bin
   SWATDIR: /usr/share/swat
   CONFIGFILE: /etc/samba/smb.conf
   LOGFILEBASE: /var/log/samba
   LMHOSTSFILE: /etc/samba/lmhosts
   LIBDIR: /usr/lib
   MODULESDIR: /usr/lib/samba
   SHLIBE1T: so
   LOCKDIR: /var/lib/samba
   STATEDIR: /var/lib/samba
   CACHEDIR: /var/lib/samba
   PIDDIR: /var/run
   SMB_PASSWD_FILE: /var/lib/samba/private/smbpasswd
   PRIVATE_DIR: /var/lib/samba/private

Several of the paths in Listing 1 point to locations with TDB files. Fortunately, many are the same, which reduces the places you need to look. For example, LOCKDIR, STATEDIR, and CACHEDIR all point to /var/lib/samba, and the PRIVATE_DIR is in a subdirectory called private. You can go into those directories and see the files for yourself.

TDB files can be broadly classified into two groups: those that are persistent and those that are temporary. Table 1 shows the names and functions of the persistent files.

The TDB files shown in Table 1 should be backed up. The procedure for doing so is provided later in this article.

In addition, there are many temporary TDB files that store state information and cached data. It is not necessary to back them up, because they are rebuilt when Samba starts.

Back to top

Using TDB files

Samba comes with three tools for manipulating TDB files:

• tdbdump: Print the contents of a TDB file.
• tdbbackup: Back up and validate TDB files.
• tdbtool: Create, view, and modify TDB files.

Peeking inside a TDB file

The fastest way to see what's in a TDB file is to dump it with the tdbdump command. Listing 2 shows a dump of the ntprinters.tdb file.

	
[root@bob ~]# tdbdump /var/lib/samba/ntprinters.tdb 
{
key(21) = "GLOBALS/c_setprinter\00"
data(4) = "\00\00\00\00"
}
{
key(13) = "SECDESC/test\00"
data(140) = "\80\00\00\00\00\00\02\00\80\00\00\00\01\00\04\80\14\00\00\00$\00\00
\00\00\00\00\004\00\00\00\01\02\00\00\00\00\00\05 \00\00\00 \02\00\00\01\02\00\00
...
\00 \02\00\00\00\02\18\00\0C\00\0F\10\01\02\00\00\00\00\00\05 \00\00\00 \02\00\00"
}
{
key(17) = "SECDESC/cups-pdf\00"
data(140) = "\80\00\00\00\00\00\02\00\80\00\00\00\01\00\04\80\14\00\00\00$\00\00
\00\00\00\00\004\00\00\00\01\02\00\00\00\00\00\05 \00\00\00 \02\00\00\01\02\00\00
...
\00 \02\00\00\00\02\18\00\0C\00\0F\10\01\02\00\00\00\00\00\05 \00\00\00 \02\00\00"
}

From the output of Listing 2, you can see that there are three keys in the database. The first key is 21 bytes long (the byte length is shown in parentheses [()]) and is called GLOBALS/c_setprinter followed by a NULL, which is ASCII zero. Non-printable characters are shown in hexadecimal format, which is a backslash followed by two hexadecimal characters. The value for the first key is 4 bytes long and it is entirely NULLs.

The next two keys are called SECDESC/test and SECDESC/cups-pdf and are both NULL terminated. The data is entirely binary and unprintable, so it has been printed out with the non-printable characters in hexadecimal format.

Backing up and restoring TDB files

Table 1 showed several TDB files that persisted across reboots and that should be backed up. Like most databases, you can't just copy the file, because the copy might become corrupted. Backup corruption happens when you copy a file as it is being written so that you have a backup that is in an inconsistent state. One alternative would be to shut down the Samba daemon, and then copy the files.

The easiest way to back up TDB files is to use the tdbbackup utility that comes with Samba. This utility safely copies a TDB file even if it is actively being written to. Another helpful feature of tdbbackup is that it can verify that a TDB file is free of corruption and automatically revert to a backup file if corruption was found. Listing 3 shows a TDB file being backed up.

	
[root@bob samba]# ls -l account_policy.*
-rw------- 1 root root 8192 Apr  7  2008 account_policy.tdb
[root@bob samba]# tdbbackup account_policy.tdb 
[root@bob samba]# ls -l account_policy.*
-rw------- 1 root root  8192 Apr  7  2008 account_policy.tdb
-rw------- 1 root root 36864 Dec  8 21:42 account_policy.tdb.bak
[root@bob samba]# tdbdump account_policy.tdb | md5sum
53ea608f0d93061480549c511756b778  -
[root@bob samba]# tdbdump account_policy.tdb.bak | md5sum
53ea608f0d93061480549c511756b778  -

The first command in Listing 3 simply lists all the files starting with account_policy to confirm that there is only one copy. Next, the account policy database is backed up by running tdbbackup account_policy.tdb. The third command looks for any files starting with account_policy to show that a new file with a .bak extension has been created. The size of the file is different from the original, but dumping each file and calculating the MD5 hash shows that their checksums are the same. The larger file size is therefore not a problem, as the contents of each key-value pair are identical.

If for some reason the original account_policy.tdb file were to become corrupted, such as during an unclean shutdown of the system, you can restore the old file from its backup. Listing 4 shows this procedure.

	
[root@bob samba]# ls -l account_policy.tdb
-rw------- 1 root root 1213 Dec  8 21:49 account_policy.tdb
[root@bob samba]# tdbbackup -v account_policy.tdb
tdb_oob len 1256 beyond eof at 1213
restoring account_policy.tdb
[root@bob samba]# ls -l account_policy.tdb*
-rw------- 1 root root 36864 Dec  8 21:49 account_policy.tdb
-rw------- 1 root root 36864 Dec  8 21:42 account_policy.tdb.bak
[root@bob samba]# tdbbackup -v account_policy.tdb
account_policy.tdb : 17 records
[root@bob samba]# tdbdump account_policy.tdb | md5sum
53ea608f0d93061480549c511756b778  -

The first command in Listing 4 shows that the file size has been cut significantly. The tdbbackup command is run again but with the -v flag, which verifies a TDB file. If the file is corrupted, you will see a summary of the errors followed by a note that the file has been restored. Comparing the file sizes with the ls command, you see that the backup was used to replace the current database.

You can safely run the tdbbackup command several times. Running it on a valid database file, like the penultimate command in Listing 4, gives a count of the records in the file. The MD5 sum also matches what was seen before the corruption.

Note: The tdbbackup command accepts wildcards, so you can back up and verify several files at the same time.

Changing TDB files

The tdbtool utility lets you make changes to the data inside a TDB file. This utility also inspects the individual keys and values within the file rather than having to dump the entire file and sort through the output.

tdbtool can accept commands on the command line, or you can open an interactive console. To accomplish your task on the command line, run tdbtool example.tdb command options, where example.tdb is the name of your file, command is the command, and the options to the command go at the end. To use the tdb shell, just run tdbtool by itself or pass the name of the file on the command line.

To create a database, run tdbtool by itself, and then enter create test.tdb. Doing so creates a database on disk called test.tdb and opens it so that any changes you make in this session will be on that file. If you have an existing TDB file, you can specify that name on the command line or use the open command. Curiously, the only way to create a TDB file directly from the command line involves specifying the name twice, such as tdbtool test.tdb create test.tdb, which returns an error but successfully creates a database.

When you have created the database or opened an existing one, the following options are available to you:

• dump: Show everything in the database, just like tdbdump.
• keys: Show just the keys in ASCII or, with the hexkeys command, show them in hexadecimal format.
• erase: Delete everything in the database without prompting.
• info: Show a summary of the number of keys and bytes used by the database.
• check: Check to see whether the database is valid.
• speed: Perform tests to see how fast data can be read and written to the database.
• show key: Print the value stored with the key.
• delete key: Delete the key and the value.

To add and manipulate data, you have the insert and store commands. Each accepts parameters for a key and a value, with the value being optional. If you don't specify a value, you will see the key with zero bytes of data are stored in the value.

Inserting a record means to create a new one, while storing a record can overwrite an existing entry. Listing 5 shows the difference between the two commands.

	
tdb> insert mykey myvalue
tdb> insert mykey newvalue
insert failed
tdb> store mykey newvalue
Storing key:

key 5 bytes
mykey
data 8 bytes
[000] 6E 65 77 76 61 6C 75 65                           newvalue
tdb> store newkey someothervalue
Storing key:

key 6 bytes
newkey
data 14 bytes
[000] 73 6F 6D 65 6F 74 68 65  72 76 61 6C 75 65        someothe rvalue

The sequence of events in Listing 5 is:

1. Insert a key called mykey with a value of myvalue. This operation succeeds.
2. Insert a key with the same name as before but with a new value. This operation fails, because the key already exists.
3. Store the key instead of inserting it. This operation succeeds, and you are given more verbose output.
4. Store a new key with a new value. This operation succeeds even though the key doesn't exist.

You can use transactions in the tdb shell to allow you to run a series of commands and apply them as a group or cancel them as a group. Listing 6 shows two transactions.

	
tdb> transaction_start
tdb> insert somekey somevalue
tdb> show somekey

key 7 bytes
somekey
data 9 bytes
[000] 73 6F 6D 65 76 61 6C 75  65                       somevalu e
tdb> transaction_cancel
tdb> show somekey
fetch failed
tdb> transaction_start
tdb> insert somekey somevalue
tdb> transaction_commit
tdb> show somekey

key 7 bytes
somekey
data 9 bytes
[000] 73 6F 6D 65 76 61 6C 75  65                       somevalu e

The transaction is first started with the transaction_start command. Next, a key is inserted. If another process were to be reading the database, it would not see this key, because it is not committed. The process with the transaction open does see the key. The transaction is then canceled with transaction_cancel. The key no longer appears.

The process is then re-run, but the transaction is committed with transaction_commit. The key then exists to all readers.

Other readers may be blocked if you are in a transaction, which means that they will hang until the transaction is finished. So be careful when using transactions on production databases! They are a great safety feature but can potentially hurt performance if overused.

Back to top

Beyond TDB

User accounts can be stored in a variety of places depending on your configuration, and two tools give you a command-line interface while taking care of the back-end communication. For example, you can use the same commands even if your user data is stored in LDAP instead of TDB files.

Using smbpasswd

The smbpasswd utility can add and delete user or machine accounts and change passwords. It is most commonly used to perform the latter task, either to change the current user's password or by root to change another user's password.

In later articles, you will learn about the different password back ends in more depth, but at a high level, the Samba passwords can be stored in several different ways depending on the vintage of Samba and which systems you're integrating with. Both smbpasswd and pdbedit (described next) can perform their actions against any back end, even if it isn't TDB.

Microsoft clients pass passwords around the network as a Microsoft-specific hash rather than plain text or a UNIX password hash. This means that it is not possible to take a Microsoft password hash and determine whether the password is the same as the password stored in the UNIX password database. Therefore, Samba must keep a separate password database for the Microsoft hashes; this is referred to as the password back end.

Using pdbedit

pdbedit manages the Samba user database and the account policies. It can do anything that smbpasswd can plus manage policies and migrate accounts between different back ends.

To show all the users in the database, run pdbedit -L. You can get even more details about your users by passing the -v flag, which is shown in Listing 7.

	
[root@bob tmp]# pdbedit -L -v
---------------
Unix username:        sean
NT username:
Account Flags:        [U          ]
User SID:             S-1-5-21-2287037134-1443008385-640796334-1001
Primary Group SID:    S-1-5-21-2287037134-1443008385-640796334-513
Full Name:            Sean
Home Directory:       \\bob\sean
HomeDir Drive:
Logon Script:
Profile Path:         \\bob\sean\profile
Domain:               BOB
Account desc:
Workstations:
Munged dial:
Logon time:           0
Logoff time:          never
Kickoff time:         never
Password last set:    Mon, 24 May 2010 21:28:49 CDT
Password can change:  Mon, 24 May 2010 21:28:49 CDT
Password must change: never
Last bad password   : 0
Bad password count  : 0
Logon hours         : FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

You must be logged in as root for the pdbedit command to work. You can edit various user settings, such as last login time, password changes, and home directories, as if you were using the Microsoft utilities.

Each user parameter has its own command-line option, so consult the pdbedit(8) manpage for the specifics.

It is also important to note that Samba treats machine accounts and user accounts in a similar manner. Listing 8 shows the password database from a Samba server configured as a domain controller.

	
[root@sergeant ~]# pdbedit -L
root:0:root
mythupstairs$:4294967295:MYTHUPSTAIRS
BOB$:1043:Machine
sean:1002:Sean,,,
sergeant$:4294967295:Machine

The names ending with a dollar sign ($) are machine accounts and are used to authenticate a machine to the domain. The corresponding secret would be stored in secrets.tdb on the remote server and passdb.tdb on the domain controller.

Back to top

Moving forward

This is the end of the Concepts, Architecture, and Design topic. In the next article, you will begin Topic 311 by learning how to download the Samba source code and compile it.

Resources

Learn

• Read about the different password back ends, including more information about how smbpasswd and pdbedit work.
  
  
• Learn about NT LAN Manager protocols, and understand a bit more about what's being passed on the wire as a user authenticates.
  
  
• Chapter 41 of the Samba manual has many details about the TDB databases and how to troubleshoot problems.
  
  
• At the LPIC Program site, find detailed objectives, task lists, and sample questions for the three levels of the LPI's Linux systems administration certification. In particular, look at the LPI-302 detailed objectives and the tasks and sample questions.
  
  
• Review the entire LPI exam prep series on developerWorks to learn Linux fundamentals and prepare for systems administrator certification based on LPI exam objectives prior to April 2009.
  
  
• Exam Preparation Resources for Revised LPIC Exams provides a list of other certification training resources maintained by LPI.
  
  
• In the developerWorks Linux zone, find hundreds of how-to articles and tutorials, as well as downloads, discussion forums, and a wealth of other resources for Linux developers and administrators.
  
  
• Stay current with developerWorks technical events and webcasts focused on a variety of IBM products and IT industry topics.
  
  
• Attend a free developerWorks Live! briefing to get up-to-speed quickly on IBM products and tools, as well as IT industry trends.
  
  
• Watch developerWorks on-demand demos ranging from product installation and setup demos for beginners, to advanced functionality for experienced developers.
  
  
• Follow developerWorks on Twitter, or subscribe to a feed of Linux tweets on developerWorks.
  
  

Get products and technologies

• Download Samba, and keep on top of the latest developments to the software.
  
  
• Download the TDB source code if you're interested in using TDB in one of your projects.
  
  
• Look at account-management tools for Samba if you need a graphical interface or better integration with different environments.
  
  
• Evaluate IBM products in the way that suits you best: Download a product trial, try a product online, use a product in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement Service Oriented Architecture efficiently.
  
  

Discuss

• Get involved in the My developerWorks community. Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.
  
  

About the author

Sean Walberg is a network engineer and the author of two books on networking. He has worked in several industries, including health care and media.

Rate this article

Comments

Back to top

Table of contents

• Overview
• Prerequisites
• Understanding TDB files
• Using TDB files
• Beyond TDB
• Moving forward
• Resources
• About the author
• Comments

Dig deeper into Linux on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers