Contents
- Introduction
- Use file name completion
- Use history expansion
- Reuse previous arguments
- Manage directory navigation with pushd and popd
- Find large files
- Create temporary files without an editor
- Use the curl command-line utility
- Make the most of regular expressions
- Determine the current user
- Process data with awk
- Conclusion
- Downloadable resources
- Related topics
- Comments
Learn 10 more good UNIX usage habits
Become a UNIX command-line wizard
Let's face it: Bad habits are hard to break. But habits that you've just become comfortable with can be even more difficult to overcome. Sometimes, a fresh look at things may provide you with an "A-ha, I didn't know you could do that!" moment. Building on Michael Stutz's excellent article, "Learn 10 good UNIX usage habits," this article suggests 10 more UNIX command-line commands, tools, and techniques that may make you more productive as a UNIX command-line wizard.
The 10 additional good habits you should adopt are:
- Use file name completion.
- Use history expansion.
- Reuse previous arguments.
- Manage directory navigation with
pushd
andpopd
. - Find large files.
- Create temporary files without an editor.
- Use the
curl
command-line utility. - Make the most of regular expressions.
- Determine the current user.
- Process data with
awk
.
Use file name completion
Wouldn't it be great if you didn't have to type a long, convoluted file name at the command prompt? Well, you don't, as it turns out. You can configure the most popular UNIX shells for file name completion, instead. This functionality works a bit differently in each shell, so I show you how to use file name completion in the most popular shells. File name completion allows you to type faster and avoid errors. Lazy? Perhaps. More efficient? Definitely!
Which shell am I running?
What happens if you donât know which shell you're currently running? Although
this trick isn't officially part of the 10 more good habits, it's still pretty
useful. As shown in Listing 1, you can use the
echo $0
or ps âp $$
command to display the shell you're using. In my case, I'm running the
Bash shell.
Listing 1. Determine your shell
$ echo $0 -bash $ ps âp $$ PID TTY TIME CMD 6344 ttys000 0:00.02 âbash
C shell
The C
shell supports the most straightforward file
name completion. Setting the filec
variable
enables the functionality. (You can use the command set filec
).
After you start typing the name of a file, you can click Escape, and the shell
fills in the name of the file—or as much as it can. For example, say
you have files named file1, file2, and file3. If you type
f
, then click Escape, file will be filled
out, and you'll have to type the 1
, 2
,
or 3
to complete the appropriate file name.
Bash
The Bash shell also provides file name completion but uses the Tab key instead
of the Escape key. You don't need to set anything to enable file name
completion in the Bash shell; it's set by default. Bash also implements an
additional feature. After typing a portion of a file name, then clicking Tab,
if you reach that point at which multiple files satisfy your request and you
need to add text to select one of the files, you can click Tab twice more for
a list of the files that match what you have typed so far. Using the earlier
examples of files named file1, file2, and file3, start by typing
f
. When you click Tab once, Bash completes
file; clicking Tab one more time expands the list of file1 file2 file3.
Korn shell
For Korn shell users, file name completion depends on the value of the
EDITOR
variable. If EDITOR
is set to vi, you type part of name, and then click Escape followed by
a backslash (\
) character. If
EDITOR
is set to emacs, you type part of
the name, and then click the Escape key twice to complete the file name.
Use history expansion
What happens if you're using the same file name for a series of commands? Well,
there's a shortcut that can quickly retrieve the last file name you used. As shown
in Listing 2, the !$
command
returns the file name that the previous command used. The file
this-is-a-long-lunch-menu-file.txt is searched for occurrences of the word
pickles. After searching, the vi
command is used
to edit the this-is-a-long-lunch-menu-file.txt file without the need for retyping the
file name. You use the bang, or exclamation point (!
),
to access the history, and the dollar sign ($
) returns
the last field of the previous command. It's a great tool if you are using long file
names repeatedly.
Listing 2. Using !$ to retrieve the last file name used with a command
$ grep pickles this-is-a-long-lunch-menu-file.txt pastrami on rye with pickles and onions $ vi !$
Reuse previous arguments
The !$
command returns the last argument used
with a command. But what happens if you have a command that used arguments
and you want to reuse just one of them? The !:1
operator returns the argument used in a command. The example in Listing 3
shows how you can use this operator in combination with the !$
operator. In the first command, a file is renamed to a more
meaningful name, but to preserve use of the original file name, a symbolic
link is created. The file kxp12.c
is renamed in a more readable manner, then the link command is used to create a symbolic link back to the original file name, in case it's still used elsewhere. The !$
operator returns the file_system_access.c
argument, and the !:1
operator returns the kxp12.c
argument, which is the first argument of the previous command.
Listing 3. Using !$ and !:1 in combination
$ mv kxp12.c file_system_access.c $ ln âs !$ !:1
Manage directory navigation with pushd and popd
UNIX supports a wide variety of directory-navigation tools. Two of my favorite
productivity tools are pushd
and popd
.
You're certainly aware that the cd
command changes
your current directory. What happens if you have several directories to navigate,
but you want to be able to quickly return to a location? The pushd
and popd
commands create a virtual directory stack,
with the pushd
command changing your current
directory and storing it on the stack, and the popd
command removing the directory from the top of the stack and returning you to
that location. You can use the dirs
command to display
the current directory stack without pushing or popping a new directory.
Listing 4 shows how you can use the pushd
and popd
commands to quickly navigate the directory
tree.
Listing 4. Using pushd and popd to navigate the directory tree
$ pushd . ~ ~ $ pushd /etc /etc ~ ~ $ pushd /var /var /etc ~ ~ $ pushd /usr/local/bin /usr/local/bin /var /etc ~ ~ $ dirs /usr/local/bin /var /etc ~ ~ $ popd /var /etc ~ ~ $ popd /etc ~ ~ $ popd ~ ~ $ popd
The pushd
and popd
commands
also support parameters to manipulate the directory stack. Using the
+n
or -n
parameter, where n is a number, you can rotate the stack left or right,
as shown in Listing 5.
Listing 5. Rotating the directory stack
$ dirs /usr/local/bin /var /etc ~ ~ $ pushd +1 /var /etc ~ ~ /usr/local/bin $ pushd -1 ~ /usr/local/bin /var /etc ~
Find large files
Need to find out where all your free disk space went? Here are a couple of tools you
can use to manage your storage. As shown in Listing 6, the
df
command shows you the total number of blocks
used on each available volume and the percentage of free space.
Listing 6. Determining volume usage
$ df Filesystem 512-blocks Used Available Capacity Mounted on /dev/disk0s2 311909984 267275264 44122720 86% / devfs 224 224 0 100% /dev fdesc 2 2 0 100% /dev map -hosts 0 0 0 100% /net map auto_home 0 0 0 100% /home
Want to find the largest files? Use the find
command
with the -size
parameter. Listing 7
shows how to use the find
command to find files
larger than 10MB. Note that the -size
parameter
takes a size in kilobytes.
Listing 7. Find all files larger than 10MB
$ find / -size +10000k âxdev âexec ls âlh {}\;
Create temporary files without an editor
This is a simple one: You need to quickly create a simple temporary file but don't
want to fire up your editor. Use the cat
command
with the >
file-redirection operator. As shown
in Listing 8, using the cat
command without a file name simply echoes anything typed to standard input;
the >
redirection captures that to the specified
file. Note that you must provide the end-of-file character when you're finished
typing—typically, Ctrl-D.
Listing 8. Quickly create a temporary file
$ cat > my_temp_file.txt This is my temp file text ^D $ cat my_temp_file.txt This is my temp file text
Need to do the same thing but append to an existing file instead of creating a new one?
As shown in Listing 9, use the >>
operator, instead. The >>
file-redirection
operator appends to an existing file.
Listing 9. Quickly append to a file
$ cat >> my_temp_file.txt More text ^D $ cat my_temp_file.txt This is my temp file text More text
Use the curl command-line utility
I can access the Web from the command line? Are you crazy? No, it's just
curl
! The curl
command lets
you retrieve data from a server using the HTTP, HTTPS, FTP, FTPS, Gopher, DICT,
TELNET, LDAP, or FILE protocols. As shown in Listing 10, I can
use the curl
command to access the current local
conditions of the National Weather Service for my location (Buffalo, NY). When
combined with the grep
command, I can retrieve the
conditions in Buffalo. Use the -s
command-line option
to suppress curl
processing output.
Listing 10. Retrieve the current weather conditions with curl
$ curl âs http://www.srh.noaa.gov/data/ALY/RWRALY | grep BUFFALO BUFFALO MOSUNNY 43 22 43 NE13 30.10R
As shown in Listing 11, you can also use the
curl
command to download HTTP-hosted files. Use the
-o
parameter to specify where the output is saved.
Listing 11. Use curl to download HTTP-hosted files
$ curl -o archive.tar http://www.somesite.com/archive.tar
This is really just a hint of what you can do with curl
. You
can start exploring a bit more simply by typing man curl
at your command prompt to display the complete usage information for the
curl
command.
Make the most of regular expressions
Many UNIX commands use regular expressions as arguments. Technically speaking, a
regular expression is a string (that is, a sequence of characters composed of
letters, numbers, and symbols) that represents a pattern defining zero or more strings.
A regular expression uses meta-characters (for example, the asterisk
[*
] and question mark [?
]
symbols) to match parts of or whole other strings. A regular expression doesn't have
to contain wildcards, but wildcards can make regular expressions useful for searching
for patterns and manipulating files. Table 1 shows some basic
regular expression sequences.
Table 1. Regular expression sequences
Sequence | Description |
---|---|
Caret (^ ) | Matches the expression at the start of a line, as in
^A |
Question mark (? ) | Matches the expression at the end of a line, as in A? |
Backslash (\ ) | Turns off the special meaning of the next character, as in
\^ |
Brackets ([] ) | Matches any one of the enclosed characters, as in [aeiou]
(Use a hyphen [- ] for a range, as in
[0-9] .)
|
[^ ] | Matches any one character except those enclosed in brackets, as in
[^0-9] |
Period (. ) | Matches a single character of any value except end of line |
Asterisk (* ) | Matches zero or more of the preceding characters or expressions |
\{x,y\} | Matches x to y occurrences of the preceding |
\{x\} | Matches exactly x occurrences of the preceding |
\{x,\} | Matches x or more occurrences of the preceding |
Listing 12 shows some of the basic regular expressions used
with the grep
command.
Listing 12. Using regular expressions with grep
$ # Lists your mail $ grep '^From: ' /usr/mail/$USER $ # Any line with at least one letter $ grep '[a-zA-Z]' search-file.txt $ # Anything not a letter or number $ grep '[^a-zA-Z0-9] search-file.txt $ # Find phone numbers in the form 999-9999 $ grep '[0-9]\{3\}-[0-9]\{4\}' search-file.txt $ # Find lines with exactly one character $ grep '^.$' search-file.txt $ # Find any line that starts with a period "." $ grep '^\.' search-file.txt $ # Find lines that start with a "." and 2 lowercase letters $ grep '^\.[a-z][a-z]' search-file.txt
Many books have been written just about regular expressions. For a more in-depth look at command-line regular expressions, I suggest the developerWorks article, "Speaking UNIX, Part 9: Regular expressions."
Determine the current user
At times, you may have an administrative script that you want to make sure a certain
user has or has not executed. To find out, you can use the whoami
command to return the name of the current user. Listing 13
shows the whoami
command run on its own;
Listing 14 shows an excerpt from a Bash script using
whoami
to make sure the current user isn't root.
Listing 13. Using whoami from the command line
$ whoami John
Listing 14. Using whoami in a script
if [ $(whoami) = "root" ] then echo "You cannot run this script as root." exit 1 fi
Process data with awk
The awk
command always seems to live in the shadows of
Perl, but it can be a quick, useful tool for simple command-line-based data
manipulation. Listing 15 shows how to get started with the
awk
command. To get the length of each line in the file
text, use the length()
function. To see if the string
ing is present in the file text, use the index()
function, which returns the location of the first occurrence of ing so that
you can use it for further string processing. To tokenize (that is, split a
line into word-length pieces) a string, use the split()
function.
Listing 15. Basic awk processing
$ cat text testing the awk command $ awk '{ i = length($0); print i }' text 23 $ awk '{ i = index($0,âingâ); print i}' text 5 $ awk 'BEGIN { i = 1 } { n = split($0,a," "); while (i <= n) {print a[i]; i++;} }' text testing the awk command
Printing specified fields of text file is a simple awk
task.
In Listing 16, the sales file consists of each
salesperson's name followed by a monthly sales figure. You can use the
awk
command to quickly total the sales for each
month. By default, awk
treats each comma-separated
value as a different field. You use the $n
operators to access each individual field.
Listing 16. Using awk for data summarization
$cat sales Gene,12,23,7 Dawn,10,25,15 Renee,15,13,18 David,8,21,17 $ awk -F, '{print $1,$2+$3+$4}' sales Gene 42 Dawn 50 Renee 46 David 46
The awk
command can be complex and used in a wide
variety of situations. To explore the awk
command
more fully, start with the command man awk
in addition to the resources mentioned in the Related topics.
Conclusion
Becoming a command-line wizard takes a bit of practice. It's easy to keep doing things the same way simply because you're used to it. Expanding your command-line resources can provide a big increase in your productivity and propel you toward becoming a UNIX command line wizard!
Downloadable resources
Related topics
- UNIX tips: Read other articles in the "UNIX tips" series.
-
"Common threads: Awk
by example, Part 1," "Common
threads: Awk by example, Part 2," "Common
threads: Awk by example, Part 3" (Daniel Robbins, developerWorks, 2001-2002):
Check out a good introduction to using
awk
. - "Speaking UNIX, Part 9: Regular expressions" (Martin Streicher, developerWorks, 17 Apr 2007): Learn to use regular expressions to discern the relevant from the irrelevant.
- New to AIX and UNIX: New to IBM® AIX® and UNIX? Visit the this page to learn more.