System Administration Toolkit

Get the most out of zsh

Content series:

This content is part # of # in the series: System Administration Toolkit

Stay tuned for additional content in this series.

This content is part of the series:System Administration Toolkit

Stay tuned for additional content in this series.

About this series

The typical UNIX® administrator has a key range of utilities, tricks, and systems he or she uses regularly to aid in the process of administration. There are key utilities, command-line chains, and scripts that are used to simplify different processes. Some of these tools come with the operating system, but a majority of the tricks come through years of experience and a desire to ease the system administrator's life. The focus of this series is on getting the most from the available tools across a range of different UNIX environments, including methods of simplifying administration in a heterogeneous environment.

zsh background

Shells under UNIX and Linux® typically fall into one of two categories based on the original shells included with the earliest versions of UNIX. The two types are the Bourne shell and the C shell; the latter being distinctive because its format and structure is like that of the C programming language.

The Bourne shell is easier to use and understand than the C shell, but it is less practical for the complex script programming that you might want to achieve within the shell programming environment. The Korn shell provides ease of use of the Bourne shell and added extensions for job control (allowing you to easily manage multiple background jobs), command-line editing and history, and added elements of the C shell to make programming easier.

The Z shell (zsh) was designed with interactive use in mind rather than programming, so it incorporates a lot of functionality that makes using and running commands much easier. Examples include more extensive filename matching (globbing), multiple I/O streams for redirecting input and output, and a fully customizable command-line completion system.

Filename generation

Filename globbing is the process behind turning a filename or file specification into a list of files for use on the command line, for example, when copying or moving files. Basic filename globbing includes the use of the ? for a single character, or * for one or more characters.

For example, to list all of the C source files, you might use Listing 1.

Listing 1. Listing all of the C source files
$ ls *.c
barney.c        betty.c         fred.c          wilma.c

And you can use letter collections (as you might in a regular expression), for example, to list the files with a "c" or "o" extension, as shown in Listing 2.

Listing 2. Listing the files with a "c" or "o" extension
$ ls *.[co]
barney.c        betty.c         fred.o
barney.o        fred.c          wilma.c

With zsh, the same wildcards work, but you can also use extended globbing for additional options. To switch on extended globbing, set the EXTENDEDGLOB environment variable, or use: $ setopt extendedglob.

You can now use constructs like the ^ character to display all of the files that don't match a given file specification. For example, to list all of the files that don't match the *.c specification, use Listing 3.

Listing 3. Listing files that don't match *.c
zsh$ ls ^*.c            
barney.o        betty.h         fred.h          fred.o

The expression <x-y> matches a range of integers. For example, if you have filed your access logs using dates, you can then select all of the files within a specific range of dates. To achieve this, name the files with the year, month, and date, in that order, using zeros to pad out the values. For example, to list the logs between the third and tenth of November 2006, you might use Listing 4.

Listing 4. Listing the logs between the third and tenth of November 2006
zsh$ ls access<20061103-20061110>.log
access20061103.log      access20061106.log      access20061109.log
access20061104.log      access20061107.log      access20061110.log
access20061105.log      access20061108.log

You can also use regular expression-like groups to match files. For example, use Listing 5 to list all the files called fred and barney.

Listing 5. Listing all the fred and barney files
zsh$ ls (fred|barney)*
barney.c        fred.c          fred.o
barney.o        fred.h

Subdirectories can also be searched by using **/; the process is recursive, such as the find command, so that you can produce the equivalent of a find command, such as $ find . -name "*.c".

In zsh (with extended globbing), this can be extended to: zsh$ ls **/*.c.

Or you can combine further examples, such as find all of the C source and compiled object files in subdirectories using the command in Listing 6.

Listing 6. Finding all C source and compiled object files
zsh$ ls **/*.(c|o)
glob/barney.c   glob/betty.c    glob/fred.o
glob/barney.o   glob/fred.c     glob/wilma.c

Finally, zsh supports a number of postfix qualifiers. To get a list of executable files, use the postfix qualifier (*) at the end of the globbing expression (see Listing 7).

Listing 7. Getting a list of executable files
zsh$ ls *(*)
barney  fred

You can also get a list of executable files using (x) at the end of the globbing expression (see Listing 8).

Listing 8. Using (x) to get a list of executable files
zsh$ ls *(x) 
barney  fred

The above list files are executable by the file's owner. A capital X selects files that are executable by others. The same also applies with the R/r (readable) and W/w (writable) mode on the file.

Command or process substitution

Within most shell environments, you can use basic process substitution to embed the output of one command into the input or arguments of another. You can do this with the backtick operator (see Listing 9).

Listing 9. Using the backtick operator to perform process substitution
$ emacs `find . -name "*.html"`

There are a number of options available to you within zsh. The primary method is to use the $() construct. This provides a direct alternative to the backticks, and it has the benefit of being more easily embeddable and nestable in certain command combinations. For example, you could rewrite Listing 9 as Listing 10.

Listing 10. Alternative to using backticks to perform process substitution
zsh$ emacs $(ls **/*.html)

Here the process substitution runs the directory listing components and returns a list of filenames, which is in turn supplied to the argument list of the emacs command.

Another useful construct is the =(list) structure. When using this feature, the element in the parentheses generates a temporary file, and the name of this file is returned. For example, you can generate a text file using Listing 11.

Listing 11. Generating a text file
zsh$ cat =(print -l tom dick harry)                            

More usefully, you can combine it with other elements in order to support more complex output and filtering. For example, you can get a list of processes matching imapd and httpd (IMAP mail service and Apache http service) using the following command (see Listing 12).

Listing 12. Getting a list of processes matching imapd and httpd
zsh$ ps -ax |fgrep -f =(print -l httpd imapd)
 406 ?? Ss  0:02.05 /usr/sbin/httpd
 426 ?? S   0:01.32 /usr/sbin/httpd
 429 ?? S   0:06.42 imapd: sulaco.mcslp.pri [] appleblog user.appleblog
 434 ?? S   0:57.81 imapd: sulaco.mcslp.pri [] mcarc user.mcarc
 435 ?? S   0:00.14 imapd: sulaco.mcslp.pri [] mlists user.mlists
 436 ?? S   0:00.12 imapd: sulaco.mcslp.pri [] play
 437 ?? S   0:01.16 imapd: sulaco.mcslp.pri [] mc
 507 ?? S   0:01.25 /usr/sbin/httpd

In Listing 12, the print command prints each argument on an individual line to a temporary file. The file is then used by fgrep as the list of matches against the process list. The temporary file is deleted once the command has been finished.

Because the feature creates temporary files from the output, you can use it to perform functions and sequences that would have previously required you to create a temporary file and delete it yourself. For example, to compare the list of files between two different directories, you could output a list of files in each directory to a temporary file, and then use the diff command to compare the output. With zsh, the same result can be achieved on a single command line (see Listing 13).

Listing 13. Compare the list of files between two different directories
zsh$ diff =(ls new) =(ls old)
< barney.o
< fred.o

In this example, the two substituted ls commands generate a temporary text file that is then compared inline by the diff command.

Multiple I/O streams

Related to the process substitution system is a more extensive system of redirection. In most shells, you are used to redirecting to and from files (see Listing 14).

Listing 14. Redirecting to and from files
$ crontab <newtab
$ ls > filelist

You can redirect to multiple outputs simultaneously with zsh (see Listing 15).

Listing 15. Redirecting to multiple outputs simultaneously
zsh$ ls >listone >listtwo

You can also input from multiple streams (see Listing 16).

Listing 16. Inputting from multiple streams
zsh$ sort <listone <listtwo

The pipe is an implicit redirect, so it works in much the same way. For example, within any shell, you can use the tee command to redirect a file and the standard output (see Listing 17 ).

Listing 17. Using the tee command to redirect a file and the standard output
$ ls | tee listone

You can use Listing 18 within zsh.

Listing 18. Redirecting to a file and standard output in zsh
zsh$ ls >fileone |cat

As an extension of the process substitution, you can also redirect to and from another command by using the <(list) and >(list) constructs (see Listing 19).

Listing 19. Redirecting to and from another command using < and >
zsh$ sort <(ls) <(ls /usr)      

In Listing 19, you combine the output from the two ls commands as the standard input for the sort command, which outputs an amalgamated and sorted list of files from two different directories.

A typical use for this is where you are using cut and paste to extract and recombine fields from a file into another. You would probably use a number of temporary files with a normal shell (see Listing 20).

Listing 20. Using temporary files to extract and recombine fields from one file into another
$ cut -f1 fileone >t1
$ cut -f5 filetone >t5
$ paste t1 t5

In zsh, you can accomplish this by skipping the temporary files, as shown in Listing 21.

Listing 21. Cut and paste to extract and recombine fields from a file within zsh
zsh$ paste -d: <(cut -d: -f1 /etc/passwd) <(cut -d: -f5 /etc/passwd)

Furthermore, because the redirected substitution can be easily nested, you can create complex structures, such as combining two fields from the passwd file in a different order from source, removing the comments, and then sorting them (see Listing 22).

Listing 22. Creating complex structures in zsh
zsh$ sort <(egrep -v '^#' <(paste -d: <(cut -d: -f5 /etc/passwd) <(cut -d: -f1 
/etc/passwd) ) )
:# mode.  At other times this information is handled by one or more of
Amavisd User:amavisd
Apple Events User:eppc
Application Owner:appowner
Application Server:appserver
Xgrid Agent:xgridagent
Xgrid Controller:xgridcontroller
sshd Privilege separation:sshd

You can simplify Listing 22 by looking at each element as in Listing 23.

Listing 23. Simplified process
-	<(cut -d: -f1 /etc/passwd) - Get the first field
-	<(cut -d: -f5 /etc/passwd) - Get the fifth field
-	<(paste -d: <(f5) <(f1) ) - Recombine them in a different order
-	<(egrep -v '^#' <(paste...) - Remove the comments
-	sort <(egrep ...) - Sort the standard input

File and command completion

Within some shells, zsh included, you can complete a file or command by pressing the TAB key. Let's use your current directory as an example (see Listing 24).

Listing 24. Listing of the current directory
zsh$ ls
barney          betty.c         fred            fred.o
barney.c        betty.h         fred.c          wilma.c
barney.o        fileone         fred.h

With completion, you can type the start of a filename: zsh$ cd bar. Then press the TAB key and get a full or partial completion: zsh$ cd barney.

Press TAB a second time, and you are presented with a list of available files (see Listing 25).

Listing 25. Getting a list of available files
zsh$ cd barney
barney*   barney.c  barney.o

The same completion process also works with directories and paths. zsh has a further trick up its sleeve with respect to completion.

Customizing completion

The limitation of completion in its standard form is that it is only able to complete files and paths (including commands) while you are entering them on the command line. There are, however, many additional areas where you might want to be able to complete the command without having to complete all of the typing. For example, Subversion, the source code control system, provides a number of secondary commands that you type in addition to other arguments. For example, to commit changes back to a Subversion repository, you use the commit command: $ svn commit. Or, to update, you use the update command: $ svn update.

But you have to type this manually. By using the custom completion control in zsh, you can add these subcommands to svn as part of the completion process. Completion control is a very complex (and sometimes confusing) system, but the fundamentals are fairly easy to understand.

Completion is controlled through a number of commands, but the main command is compctl. This provides a simple interface for basic completion. Completion can be applied globally (in other words, like the files and paths), or it can be applied to specific commands.

There are a range of options and formats available, but to continue with the Subversion idea, you can use the -k option to provide an array of words that act as a potential completion to the svn command (see Listing 26).

Listing 26. Using the -k option
zsh$ compctl -k '(commit checkout update status)' svn

Now when you type svn at the start of the command, you need to press TAB to complete the command (see Listing 27).

Listing 27. Pressing TAB to complete command
zsh$ svn com <TAB>
zsh$ svn commit

The completion system includes standard functions and completions -- for example, lookups of valid users on the system, home directories, hosts, networks, and so on. You can also add and extend these yourself to produce customized completion results.

For example, you can create a custom function called activeusers that returns the output from the users command: zsh$ function activeusers { reply =(`users`) }.

You can now use this as a completion for another command, for example, chat: zsh$ compctl -K activeusers chat.

Now when you type chat on the command line, you can obtain a completion list that only returns a list of the users currently logged in.

The available options and possibilities with the custom completion system are too numerous and extensive to cover here. See Related topics for the official documentation on the available options.


zsh incorporates a range of functionality designed to make the interaction between the user and shell environment easier. Extensions include better ways of substituting commands and redirecting information in and out to different processes. These options alone enable you to convert the many commands that would be needed in another shell into a single command-line entry within zsh.

The real differentiator between zsh and other shells, even the recent improvements provided in shells like bash, is the ability to customize the auto-completion system to work with more than just filenames and paths. Extending the functionality to support additional arguments to your existing commands is just one example, but the system is so flexible that you could almost complete any command or command-line element.

Downloadable resources

Related topics

Zone=AIX and UNIX
ArticleTitle=System Administration Toolkit: Get the most out of zsh