Tip: Concatenating files with cat

Get to know your textutils

Jacek Artymiak (jacek@artymiak.com), Freelance author and consultant

Jacek Artymiak works as a freelance consultant, developer, and writer. Since 1991 he's been developing software for many commercial and free variants of UNIX and BSD operating systems (AIX, HP-UX, IRIX, Solaris, Linux, FreeBSD, NetBSD, OpenBSD, and others), as well as MS-DOS, Microsoft Windows, Mac OS, and Mac OS X. Jacek specializes in business and financial application development, Web design, network security, computer graphics, animation, and multimedia. He's a prolific writer on technology subjects and the coauthor of "Install, Configure, and Customize Slackware Linux" (Prima Tech, 2000) and "StarOffice for Linux Bible" (IDG Books, 2000). Many of Jacek's software projects can be found at SourceForge. You can learn more about him at his personal Web site and contact him at jacek@artymiak.com.

Summary: In our continuing series on GNU text utilities, Jacek Artymiak takes a look at cat -- the command that UNIX lovers love to love, and UNIX haters love to hate.

Tag this!

Date: 01 Oct 2002
Level: Introductory
Activity: 9856 views

Often you need to process several files as one and save the results of such processing to a single output file. The cat (short for "concatenate") command takes one or more files on its input and prints them to its output as a single file. For example, cat chapter01 chapter02 chapter03 > book saves three chapterXX files into a single book file.

The input files are printed in the order they are listed after the cat command, so to reverse the order of information, you must first reverse the order of input files. Also, when the number of files that need to be processed is too large for you to type their names by hand, you may use a wildcard, as in cat chapter* > book remembering that the names of the files will be sorted in ascending order. This may cause interesting problems when you suddenly find out that chapter13 is sent to the output beforechapter2, while chapter13 is sent to the output afterchapter02.

When the output of cat is not redirected to a file or another command's standard input, cat behaves as most command-line tools and send its output to the console. This means that you can use cat to display files; for example, you can use cat /etc/passwd to display the contents of the system password file. For convenience, should should view large files with less, as in less /etc/passwd (you can learn more about less by typing man less).

Although cat is primarily used to concatenate files, you can also use it for simple automatic processing of input. For example, you can remove multiple blank lines with a single blank line (use the -s option), which is a good way to clean up source code before showing it to the world. Unfortunately, cat doesn't have an option for removing blank lines altogether. But that's not a big problem, because you can remove them with the handy sed command:

Listing 1. Using sed with cat to remove blank lines

$ cat -s /etc/X11/XF86Config | sed '/^[[:space:]]*$/d'

...

# Multiple FontPath entries are allowed (they are concatenated together)

# By default, Red Hat 6.0 and later now use a font server independent of

# the X server to render fonts.

    FontPath   "/usr/X11R6/lib/X11/fonts/TrueType"

    FontPath   "unix/:7100"

EndSection

...

Blank line compression is a handy trick for reading config files and sources of HTML pages, especially those generated by scripts that insert gratuitous newlines, as well as sources that contain large conditional structures whose items have been separated with empty lines.

Another important feature of cat is its ability to number lines. This functionality comes in handy for program documentation as well as legal and scientific documentation. Line numbers printed on the left side make it easy to refer to a certain part of the document. This is very important in programming, scientific research, business reporting, or even legislative work. There are two options for numbering lines: -b for numbering non-blank lines only and -n for numbering all lines:

Listing 2. Numbering lines

$ cat -b /etc/X11/XF86Config

...


    14  # Multiple FontPath entries are allowed (they are concatenated together)

    15  # By default, Red Hat 6.0 and later now use a font server independent of

    16  # the X server to render fonts.

    17      FontPath   "/usr/X11R6/lib/X11/fonts/TrueType"

    18      FontPath   "unix/:7100"

    19  EndSection
...

$ cat -n /etc/X11/XF86Config

...


    20  # Multiple FontPath entries are allowed (they are concatenated together)

    21  # By default, Red Hat 6.0 and later now use a font server independent of

    22  # the X server to render fonts.

    23  

    24      FontPath   "/usr/X11R6/lib/X11/fonts/TrueType"

    25      FontPath   "unix/:7100"

    26  

    27  EndSection

...

cat can also help when you are viewing files that contain non-printing character like tabs. You can reveal them with these options:

-T display tabs as ^I
-v displays non-printing characters, except line-feed and tab using their "control sequence" equivalents. For example, when you process a file generated on a Windows system, it will use Control-M (^M) characters to mark the end of the line. Characters with codes higher than 127 will be preceded with M- as in "meta", which is an equivalent of Alt- characters on other systems.
-E adds the dollar sign ($) at the end of each line.

Listing 3. Displaying non-printing characters

$ cat -t /etc/X11/XF86Config
...

# Multiple FontPath entries are allowed (they are concatenated together)

# By default, Red Hat 6.0 and later now use a font server independent of

# the X server to render fonts.


^IFontPath^I"/usr/X11R6/lib/X11/fonts/TrueType"

^IFontPath^I"unix/:7100"

EndSection
...


$ cat -E /etc/X11/XF86Config

...

# Multiple FontPath entries are allowed (they are concatenated together)$

# By default, Red Hat 6.0 and later now use a font server independent of$

# the X server to render fonts.$

$

    FontPath   "/usr/X11R6/lib/X11/fonts/TrueType"$

    FontPath   "unix/:7100"$

$

EndSection$

...

$ cat -v /etc/X11/XF86Config
...

^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@M-|M-8^X^@^@^@

P^@^O"M-X^O M-@^M^@^@^@M-^@^O"M-@M-k^@M-8*^@

@M-^H$M-@M-9|A(M-@)M-yM-|M-sM-*M-hW^A^@^@j^@

M-|M-sM-%1M-@M-9^@^B^@^@M-sM-+fM-^A= ^@ ^@

F^@^@   ^@M-9^@^H^@^@M-sM-$M-G^E(l!M-@M-^?

^IM-A5^@^@^D^@PM-^]M-^\X1M-H%^@^@^D^@tyM-G
...

Next time, we'll take a look at head and tail. See you then! In the meantime, if you have questions or comments, I'd love to hear from you -- send e-mail to jacek@artymiak.com.

Resources

The introduction to text utilities in this series complements information found in the man and info pages. To learn more, open a new terminal window and type man cat or info cat -- or you can open a new browser window and view the cat man page at gnu.org.
See also the Jargon File entry for cat. Learn why cat is considered both an excellent example of user interface design, and the canonical example of bad user interface design.
Find even more info on these tools we know and love as textutils in the GNU text utilities manual (an expanded view of the same TOC lives at MIT, where you can also find a great list of even more useful GNU tools).
Windows users can find these tools in the Cygwin package.
Mac OS X users may want to try Fink, which installs a rich UNIX environment under the Mac OS X.
Something just not working for you? Try checking the Frequently asked questions for GNU textutils.
Need more introductory info before delving in to the tools we've covered here? Try starting with UNIXhelp for users.
Or try the introductory article in this series "Get to know your textutils" (developerWorks, October 2002).
The classic work in this field is UNIX Power Tools, by Jerry Peek, Tim O'Reilly, and Mike Loukides (O'Reilly and Associates, 1997); ISBN 1-56592-260-3.
Find the Linux resource you're looking for in the developerWorks Linux zone.

About the author

Comments (Undergoing maintenance)

Trademarks | My developerWorks terms and conditions

Close [x]

Help: Update or add to My dW interests

What's this?

This little timesaver lets you update your My developerWorks profile with just one click! The general subject of this content (AIX and UNIX, Information Management, Lotus, Rational, Tivoli, WebSphere, Java, Linux, Open source, SOA and Web services, Web development, or XML) will be added to the interests section of your profile, if it's not there already. You only need to be logged in to My developerWorks.

And what's the point of adding your interests to your profile? That's how you find other users with the same interests as yours, and see what they're reading and contributing to the community. Your interests also help us recommend relevant developerWorks content to you.

View your My developerWorks profile

Return from help

Close [x]

Help: Remove from My dW interests

What's this?

Removing this interest does not alter your profile, but rather removes this piece of content from a list of all content for which you've indicated interest. In a future enhancement to My developerWorks, you'll be able to see a record of that content.

View your My developerWorks profile

Return from help

static.content.url=http://www.ibm.com/developerworks/js/artrating/

SITE_ID=1

Zone=Linux, Open source

ArticleID=11261

ArticleTitle=Tip: Concatenating files with cat

publish-date=10012002

author1-email=jacek@artymiak.com

author1-email-cc=