Debugging make

Tips and tricks to get make working for you, not against you

Make utilities such as GNU make, System V make, and Berkeley make, are fundamental tools for streamlining the application build process, but each one is just a little different from the others. Learn the structure of the makefile and how to avoid common mistakes in its creation, discover how to fix or work around portability issues, and pick up hints for solving other problems as they crop up.

Share:

Peter Seebach (dw@seebs.net), Writer, Freelance

Author photoPeter Seebach has been using computers for years and is gradually becoming acclimated. He still doesn't know why mice need to be cleaned so often, though.



24 October 2006

Also available in Russian Japanese

Most UNIX® and Linux® programs are built by running make. The make utility reads a file (generally named either "makefile" or "Makefile," but hereafter merely referred to as "a makefile") that contains instructions and performs various actions to build a program. In many build processes, the makefile is itself generated entirely by other software; for instance, the autoconf/automake programs are used to develop build routines. Other programs may ask you to directly edit a makefile, and of course, new development may require you to write one.

The phrase "the make utility" is misleading. There are at least three distinct variants in common use: GNU make, System V make, and Berkeley make. Each grew from a core specification from the early UNIX days, and each adds new features. This results in a difficult situation: fairly commonly used features, such as including other files in a makefile by reference, cannot be done portably! The decision to simply write a program to create makefiles is one solution. Because GNU make is free and widely distributed, some developers simply code for it; similarly, a few projects with BSD origins require you to use Berkeley make (which is also free).

Less common but still relevant are Jörg Schilling's smake, and the absentee fifth member of the family, historical make, which defines the common feature subset that all the others share. While smake is not the default make on any system, it's a good make implementation, and some programs (especially Schilling's) use it by preference.

Let's review some of the most common problems you'll encounter when working with makefiles.

Understanding the makefile

To debug make, you have to be able to read a makefile. As you know, the purpose of a makefile is to give instructions for building a program. One of make's key features is dependency management: make attempts to rebuild only what it has to when a program is updated. In general, this is expressed through a series of dependency rules. A dependency rule looks like this:

Listing 1. Form of dependency rule
target: dependencies
	instructions

The main problem people encounter when writing their first makefile is visible in this construction; or rather, invisible. The indentation is a tab. It is not any number of spaces. The Berkeley make error message for a file using spaces in this format is not terribly helpful:

Listing 2. Berkeley make error message
make: "Makefile" line 2: Need an operator
make: Fatal errors encountered -- cannot continue

GNU make, while still unable to process the file, gives a more helpful suggestion:

Listing 3. GNU make error message
Makefile::2: *** missing separator (did you mean TAB instead of 8 spaces?).  Stop.

Note that both the dependencies and the instructions are optional; only the target and the colon are required. So, that's the syntax. What are the semantics? The semantics are that, if make wishes to build target, it will first look at dependencies. In fact, it will recursively attempt to build them; if the dependencies in turn have dependencies, those will be dealt with before this rule continues. If target exists and is at least as new as all of the items listed in dependencies, nothing is done. If target does not exist, or one or more dependencies is newer, then make executes instructions. Dependencies are processed in the order that they are specified. If no dependencies are specified, the instructions are followed anyway. Dependencies are also called sources.

If a target is given on the command line (for example, make foo), then make will attempt to build that target. Otherwise, it will try to build the first target listed in the file. One convention some developers use is to have the first target look like this:

Listing 4. Commonly used first target convention
default: all

Some people assume that make uses this rule because it's named "default." Not so; it's used because it's the first rule in the file. You could name it anything you wanted, but the name "default" is a good choice because it communicates to the reader. Remember that your makefile is going to be read by humans, not just make programs.

Phony targets

In general, it is asserted that a target's function is to create a file from other files. In fact, this is not always the case. Most makefiles have at least a couple of rules that never create a target. Consider the following example rule:

Listing 5. Example phony target
all: hello goodbye fibonacci

This rule instructs make -- if it wishes to build the target all -- to first make sure that hello, goodbye, and fibonacci are up to date. Then...nothing. No instructions are provided. After this rule is complete, no file named "all" is created. The target is a fake. The technical term used in some varieties of make is "phony."

Phony targets are used for organizational purposes and are a wonderful thing in writing a clear and legible makefile. For instance, one often sees rules like this:

Listing 6. Intelligently used phony target
build: clean all install

This specifies an order of operations for the build process.

Special targets and sources

A few special targets are defined that have special effects on make, providing a configuration mechanism. The exact set varies from one implementation to another; the most common is the .SUFFIXES target, the "sources" for which are a series of patterns to be added to the list of recognized file suffixes. Special targets don't count for purposes of the usual rule that make builds the first target in the makefile by default.

Some versions of make allow special sources to be specified along with the dependencies for a given target, such as .IGNORE, which indicates that errors from commands used to build this target should be ignored, as though a dash preceded them. These flags are not especially portable but may be necessary to understand in debugging a makefile.

Generic rules

There are implicit rules in make for performing generic transforms based on filename suffixes. For instance, with no makefile present, create a file called "hello.c" and run make hello:

Listing 7. Example of implicit rule for C files
$ make hello
cc -O2   -o hello hello.c

makefiles for larger programs may simply specify a list of object modules they need (hello.o, world.o, and so on), then provide a rule for converting .c files to .o files:

Listing 8. Rule for converting .c to .o files
.c.o:
	cc $(CFLAGS) -c $<

In fact, most make utilities have a rule very much like this built in already; if you ask make to build file.o, and it has file.c, it'll do the right thing. The term "$<" is a special predefined make variable that refers to the "source" for a rule. That brings us to make variables.

Generic rules depend on the declaration of "suffixes," which make then recognizes as filename extensions rather than part of a name.

Variables

The make program uses variables to make it easier to reuse common values. The most set value is probably CFLAGS. A few things should be clarified about make variables. They are not necessarily environment variables. If no make variable with a given name exists, make will check for environment variables; however, that doesn't mean that make variables are exported to the environment. The precedence rules are arcane; in general, the order from highest to lowest precedence is this:

  1. Command-line variable settings
  2. Variables set in a parent make process's makefile
  3. Variables set in this make process's makefile
  4. Environment variables

Thus, environment variables are used only if a variable is not set in any makefile or on the command line. (Note: parent makefile variables are sometimes, but not always, passed down. The rules, as you may have guessed, vary from one flavor of make to another.)

A common problem people run into with make is variables inexplicably being replaced by parts of their names: for instance, $CFLAGS being replaced by "FLAGS". To refer to a make variable, put its name in parentheses: $(CFLAGS). Otherwise, you get $C followed by FLAGS.

A number of variables have special meanings that are a function of the rule they're being used in. The most commonly used are:

  • $< - The source from which the target is to be made
  • $* - The base name of the target (no extensions or directory)
  • $@ - The full name of the target

Berkeley make deprecates these variables, but they are (for now) still portable. Sort of portable, anyway; the exact definitions may vary between make implementations. Anything complicated you write with these will likely end up specific to a given implementation.

Shell scripting

It's occasionally desirable to perform some sort of task beyond the scope of what can be done portably in make. The conventional solution, since make runs everything through the shell, is to write an inline shell script. Here's how.

First, be aware that while shell scripts are traditionally written on multiple lines, they can be compressed to a single line with semicolons to separate statements. Second, be aware that this is illegible. The solution is a compromise: write the script with the usual indentation, but with each line ending with "; \". This ends each shell command syntactically (with a semicolon) but makes the text part of a single make command that will be passed to the shell all at once. For instance, the following might show up in a top-level makefile:

Listing 9. Breaking lines in a shell script
all:
	for i in $(ALLDIRS) ; \
	do      ( cd $$i ; $(MAKE) all ) ; \
	done

This illustrates the three things to keep in mind. First, the use of semicolons and backslashes. Second, the use of $(VARIABLE) for make variables. Third, the use of $$ to pass a dollar sign to the shell. That's it! It really is that easy.

Prefixes

By default, make prints every command it runs, and aborts if any command fails. In some cases, it's possible that a command will appear to fail, but you will want the build to continue. If the first character of a command is a hyphen (-), the remainder of the line is executed, but its exit status is ignored.

If you don't want to echo a command, prefix it with an at-sign (@). This is most commonly used for displaying messages:

Listing 10. Suppressing echo
all:
	@echo "Beginning build at:"
	@date
	@echo "--------"

Without the @ signs, this would produce the output:

Listing 11. Commands without @
echo "Beginning build at:"
Beginning build at:
date
Sun Jun 18 01:13:21 CDT 2006
echo "--------"
--------

While the @ sign doesn't really change what make does, it's a very popular feature.


Things you can't do portably

Some things that everyone wants to do cannot be portably done. But there are some workarounds to be had.

Include files

One of the most frustrating historical compatibility problems is handling inclusion in makefiles. Historical make implementations didn't always provide a way to do this, although all modern varieties appear to. The GNU make syntax is simply include file. The traditional Berkeley syntax is .include "file". At least one Berkeley make now supports the GNU notation as well, but not all do. The portable solution, discovered by both autoconf and Imake, is just to include every variable assignment you think you might want.

Some programs simply require the use of GNU make. Some require Berkeley make. Still others, smake. If you need include files badly enough, it's not unthinkable to simply specify a make utility that your tree will be built with. (Of the three portable ones distributed in source, my favorite is Berkeley make.)

Getting variables to nested builds

There is no really good way to do this. If you use an include file, you run into the portability problem of including the file cleanly. If you set the variables in every file, it becomes very hard to overwrite them all. If you set them only in a top-level file, independent builds in subdirectories will fail because the variables aren't set!

Depending on your version of make, one reasonably good solution is to conditionally set variables in every file only if they are not already set; then a change in the top-level file will affect subdirectories in a full build. Of course, then going into a subdirectory and running make there will get different and incompatible results...

This is amplified by the lack of include files, as anyone who has ever struggled with Imake's multi-thousand-line makefiles can attest.

Some writers advocate a simpler solution: don't use recursive make at all. For most projects, this is entirely feasible and can dramatically simplify (and speed up!) compilation. The article by Peter Miller, "Recursive Make Considered Harmful" (see Resources) is probably the canonical source.


What to do when there's a problem

First, don't panic. Developers have been having weird problems with make since before a complete version was written. Implicit rules, unexpected variable substitutions, and syntax errors from embedded shell scripts are just where the fun begins.

Read the error message. Is it actually from make, or is it from something make is calling? If you have a nested build, you might need to thread up through a bunch of error messages to find the actual error.

If a program isn't being found, first check to see whether it's installed. If it is, check paths; some developers have the habit of specifying the absolute path to a program in makefiles, which may fail on other systems. If you installed something in /opt, and the makefile refers to /usr/local/bin, the build will fail. Fix the path.

Check your clock; more importantly, look at the dates of files in your build tree, and other files on your system, and your clock. The behavior of make when confronted with chronologically inconsistent input data can range from harmless to surreal. If you had clock problems (some "new" files datestamped in 1970, for instance), you will need to fix those. The "touch" utility is your friend. The error messages you will get from clock skew will not generally be obvious.

If the error messages you're getting imply that you have syntax errors, or tons of variables unset or set incorrectly, try a different version of make; for instance, some programs will get cryptic errors from gmake but build fine with smake. Really weird errors may indicate that you're running a Berkeley makefile with GNU make, or vice versa. Programs native to Linux often assume GNU make and fail in inexplicable ways with anything else, even if there's no hint of this in the documentation.

Debugging flags can be pretty useful. For GNU make, the -d flag will give you a HUGE amount of information, some of it useful. For Berkeley make, the -d flag takes a set of flags; -d A is the complete set, or you can use subsets; for instance, -d vx will give debugging information about variable assignments (v) and cause all commands to be run through sh -x so the shell will echo the exact command it received. The -n debugging flag causes make to print a list of things it thinks it needs to do; it's not always correct, but it often gives you an idea of what's wrong.

Your goal in debugging a makefile is to figure out what make is trying to build, and what commands it thinks will build it. If make is picking the right commands, and they're failing, you may be done debugging make -- or you may not. For instance, if the attempt to compile a program fails due to unresolved symbols, it's possible that an earlier phase of the build process did something wrong! If you are unable to figure out what's wrong with a command, and it looks like it should work, it's quite possible that one of the files already created by make was created incorrectly.


Documentation

As is often the case, the primary documentation for GNU make is not available in man format, unfortunately; you have to use the info system instead, and you can't just run man make and look for information about it. However, the documentation is reasonably complete.

It is fairly hard to get good documentation on the "safe subset" of features that all implementations support. The Berkeley and GNU make documentation both tend to mention when they're describing an extension, but it's good to test things before relying too heavily on guesses about the exact boundaries.

Over time, drift between the BSDs has resulted in subtle differences between their implementations of make. Of the three, NetBSD is the one for which make is most actively supported on other systems; the NetBSD pkgsrc system is in use on other platforms, and relies heavily on NetBSD's implementation of make.

Resources

Learn

Get products and technologies

  • With IBM trial software, available for download directly from developerWorks, build your next development project on Linux.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Linux on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Linux, Open source
ArticleID=170226
ArticleTitle=Debugging make
publish-date=10242006