Why capture command-line input and output?

The online reference information for many command-line UNIX and Linux® commands (the man page) is the primary source of information about these commands. All man pages are supposed to explain all the options that are available for a given command and many provide examples of associated program output, including any program prompts and associated user input.

To obtain this information, someone has to create those examples. Reading the source code to see every possible prompt to the user and the scenarios it displays is usually less functional than simply running the program to see what it does. To convert this experimentation into documentation, the developers (or writers who create reference information for an application) find it useful to keep a complete record of program input, prompts for information that are displayed by an application, and user input in response to those prompts.

Capturing a user's interaction with a command-line application is not just a writing task. Most developers of command-line applications use the C programming language's printf() command (or its equivalent in other languages) as their initial debugging tool, using it to display the contents of different variables and data structures throughout an application. Depending on the amount of information displayed, it's usually easiest to capture this output in a file for subsequent examination and analysis. Similarly, system administrators often like to keep a step-by-step record of interactive system or server configuration and initialization.

The next sections of this article explore various ways of capturing user interaction with a command-line application, ranging from using features included in all UNIX and Linux shells to using specific tools to automate capturing this type of information.

Capturing output and error messages

UNIX and Linux systems make it easy to capture program output and errors in separate files using capabilities that are built into the command interpreters, known as shells, that are used on those systems. These capabilities, known as shell redirection, enable users to redirect program output (known as standard output, or stdout to its friends) and program error output (known as standard error or stderr) to specific files or other destinations by using angle bracket characters or numeric identifiers. Different shells use slightly different syntax to capture program output and error messages, as shown in the following list:

• Writing standard output from the ls program to the file1 file in the /bin/sh, /bin/bash, /bin/csh, and /bin/ksh shells:

  ls > file1

  
  
• Writing error output from the ls program to the file1 file in the /bin/sh, /bin/bash, /bin/csh, and /bin/ksh shells:

  ls 2> file1

  
  

  (There is no easy way to redirect only error messages to a file if you are using the /bin/csh shell.)

• Writing both standard and error output from the ls program to the file1 file in the /bin/sh, /bin/bash, /bin/csh, and /bin/ksh shells:

  ls &> file1

  
  
• Writing standard output from the ls program to the file1 file and error output from this command to the file2 file in the /bin/sh, /bin/bash, and /bin/ksh shells:

  ls 1> file1 2> file2

  
  

Shell redirection can be handy in capturing standard program output, error messages from a program, or both to text files. However, this isn't sufficient if what you really want to do is to capture a user's interaction with a program. For that, you can take advantage of the shell's interactive capabilities and an additional type of output redirection known as a pipe.

Capturing program interaction using an interactive shell

The previous section showed how to write specific aspects of a user's interaction to specific files, which is helpful if you're trying to capture debugging output, program usage information, and so on. However, if you're writing documentation for an application or simply keeping a record of using or experimenting with an application, you want to see what you're providing as input to the program (an example of what is known as standard input or stdin), and the application's reaction to that input. The last two sections of this article discuss some applications that are designed for that purpose. This section focuses on how you can do that using only your favorite shell, an additional type of shell redirection known as a pipe, and a standard UNIX or Linux utility known as tee.

In UNIX and Linux shells, a pipe is represented by the "|" symbol and connects the standard output of one command to the standard input of another. The UNIX or Linux tee application does what its name suggests if you're familiar with various plumbing connectors—it splits its input into two different streams. One of these remains standard output, while the other is a file whose name you specify on the command line. Therefore, you can use the combination of a pipe and the tee command to combine redirecting the output of a program into a file and see that output. For example, the following command displays both the output of the ls /etc command and captures the standard output of that command in the ls.out file:

    ls /etc | tee ls.out
  

You can also redirect standard error and standard output via a pipe on the command line by using the "|&" characters. For example, the following command both displays the output and any error messages of the ls /etc command and uses the tee command to capture the output and error messages in the ls.out file:

    ls /etc |& tee ls.out
  

You can combine redirection of standard output and standard error via a pipe, the tee command, and your favorite shell's interactive mode to capture all input, output, and error messages to a file using a command such as the following:

    bash -i |& tee output_file.txt
  

This command executes the Bash shell in interactive mode, meaning that it is essentially a shell that you can interact with until you explicitly terminate it. The command uses the "|&" characters to redirect all the output and error messages of that shell to the tee command, which both displays all output and error messages and saves a copy of everything to the output_file.txt file. When you're done executing any commands that you want to capture your interaction with, you can type the exit command (or the Control-d key combination on most systems) to terminate the shell. The output_file.txt file contains a complete record of anything you typed, any output from the programs you ran, and any error messages that those commands displayed—exactly as it looked while you were executing those commands.

Using the script command

While UNIX and Linux commands such as bash -i |& tee output_file.txt aren't really that complex, it can be a hassle to type them, remember the correct order of the "|" and "&" characters, remember the tee command, and so on. A much simpler solution that is available on most UNIX and Linux systems is the script command, which does all that for you. The script command starts a sub-shell and saves a copy of everything you type, all output from any programs that you run, and any errors that those programs generate to an output file. If you don't specify the name of the file that you want to save everything in, the script command creates a file called typescript, but you can also specify the name of the file that you want to write to on the command line. For example, the following command captures everything you type, all program output, and any error messages to the output_file.txt file:

    script output_file.txt
  

Similar to using an interactive shell, as discussed in the previous section, you exit the script command and close its output file by typing the exit command (or the Control-d key combination on most systems).

If you compare the contents of an output file produced by the interactive shell mechanism discussed in the previous section and one produced by using the script command, you immediately notice that files produced by the script command usually contain control characters, which are characters such as backspace, return, special characters used to display fancy prompts, and characters used if you edit a previous shell command. The output files created by the script command contain these characters because they contain an exact record of everything you type, not how it is interpreted by the shell. Output files produced using the interactive shell mechanism discussed in the previous section don't contain these characters, because they contain what is eventually forwarded to the shell after control characters, such as backspace and return, are processed.

It's easy enough to clean up a script output file using your favorite text editor, and a variety of cleanup scripts are available on the web. My favorite cleanup mechanism is from the Perl Monks website and uses a combination of a simple Perl command and the standard UNIX and Linux col command, which filters control characters out of its input, to clean up the output files produced by the script command. For example, to create a clean copy named output_file.txt.clean of a script output file named output_file.txt, you execute the command shown in Listing 1.

    
    cat output_file.txt | \
      perl -pe 's/\e([^\[\]]|\[.*?[a-zA-Z]|\].*?\a)//g' |\
        col -b > output_file.txt.clean
  

If you do this sort of thing often, you probably want to create a small shell script to automate this process just in case you have problems typing the Perl substitution command in the second line. Listing 2 is a simple shell script that does this.

    
    #!/bin/bash

    if [ $# -ne 2 ] ; then 
      echo "Usage: cleanup script-file output-file"
      exit
    fi

    cat $1 | \
      perl -pe 's/\e([^\[\]]|\[.*?[a-zA-Z]|\].*?\a)//g' |\
        col -b > $2
  

Shell redirection, the interactive shell command discussed in the previous section, and the script command discussed in this section are all mechanisms for capturing program interaction that you can use on almost every UNIX and Linux system. However, some UNIX systems provide additional tools for capturing program interaction. Those tools provide additional capabilities that can be useful in special situations, as discussed in the next section.

Platform-specific text-capture utilities

Most UNIX and Linux systems simply provide the script command to automate capturing program interaction, but AIX operating systems provide a handy utility, cleverly called capture, that enables you to write a specific screen's worth of text to a file. This is helpful when you want to show an example of running a command that manipulates the screen, such as a text editor. Trying to capture an example of using the vi editor to a file using shell redirection or the script command creates an output file that contains many control characters and is essentially unintelligible.

If you do not specify the name of a specific file to write to on the command line, the AIX capture command creates a file called screen.out. Starting the capture command creates a shell in which you can execute any commands that you want until the screen shows exactly what you want to capture. At that point, typing the Control-p key combination writes the contents of the screen to your output file. To terminate the capture command, simply type the exit command (or Control-d on most systems).

When using the capture command, it's important to remember that it doesn't capture the text on the screen until you type Control-p and that typing Control-p multiple times simply overwrites your output file unless you start the capture command with the -a option to tell it to append to the output file rather than overwrite it.

Summary

Capturing your interaction with a program is a surprisingly common task. Though UNIX and Linux shells provide built-in mechanisms for writing specific portions of that interaction to text files, using an interactive shell or the script command produces single output files that provide a complete sequential record of program interaction. The mechanisms for producing these output files (and cleaning them up, when necessary) are useful tools for almost any UNIX or Linux writer, developer, or system administrator.

The next article in this series will discuss standard tools for capturing images that show the entire screen or a specific window on UNIX and Linux systems. It will also explain how to convert those images into a variety of standard graphics formats.

Resources

Learn

• The AIX and UNIX developerWorks zone provides a wealth of information relating to all aspects of AIX systems administration and expanding your UNIX skills.
  
  
• New to AIX and UNIX? Visit the New to AIX and UNIX page to learn more.
  
  
• Browse the technology bookstore for books on this and other technical topics.
  
  

Get products and technologies

• The script command is installed by default on most UNIX and Linux systems, and is part of the bsdutils package on Linux systems, which is itself part of the util-linux package on Linux systems. You can get the source code for this package as part of the standard packages for your system or from ftp://ftp.us.kernel.org/pub/linux/utils/util-linux/.
  
  
• The Perl Monks Web site provided the incantation used in this article to clean up script output and has many other great tips for Perl fans.
  
  

Discuss

• Check out developerWorks blogs.
  
  
• Follow developerWorks on Twitter.
  
  
• Get involved in the My developerWorks community.
  
  
• Participate in the AIX and UNIX forums:
  • AIX Forum
  • AIX Forum for developers
  • Cluster Systems Management
  • IBM Support Assistant Forum
  • Performance Tools Forum
  • Virtualization Forum
  • More AIX and UNIX Forums
  
  

About the author

Rate this article

Comments

Back to top

Table of contents

• Why capture command-line input and output?
• Capturing output and error messages
• Capturing program interaction using an interactive shell
• Using the script command
• Platform-specific text-capture utilities
• Summary
• Resources
• About the author
• Comments

Dig deeper into AIX and UNIX on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers