While brainstorming or drinking coffee one day, you've thought up a great idea for a new networked service -- something that could change the world, or at least the Internet. Unfortunately, you don't have a clue where to start. You read some socket programming tutorials and learn all about how you could write your service as a single processing loop using the select() call to manage handling multiple connections.
But wait, that sounds a lot like co-operative multitasking, which certainly isn't the way things should work on a modern operating system. You definitely don't want to waste your time keeping track of what's going on with the various streams yourself.
Luckily for you, UNIX® lets you create child processes to do the work. Also, modern UNIXes let you use POSIX threads, too. Read on to find out what these are and see how you can use them to do several things at the same time.
Splitting your process with fork()
Before threads and lightweight processes were invented on UNIX systems, a simple system call, fork(), was used to create multiple streams of execution by splitting off an identical child process like binary osmosis. These "forked" processes share the same code, but get a private copy of the original's current data, including open files, sockets, security privileges, and any other artifacts the implementation deemed fit.
This sounds like a great solution to the problem of needing to handle multiple network connections simultaneously; just fork() off a child process after you've accepted a connection and let it handle protocol negotiation and data transfers while the original process waits for a new connection. See Figure 1 below.
Figure 1. Forked processes
Listing 1 shows how a program using fork() is laid out.
Listing 1. General
fork() program layout
#include <unistd.h>
#include <sys/types.h>
int main( int argc, char **argv )
{
pid_t child = 0;
child = fork();
if( child < 0 ) {
/* An error has occurred! */
handle_error();
} else if( child == 0 ) {
/* Parent process. */
handle_parent_duties();
} else {
/* Child process. */
handle_child_duties();
}
return 0;
}
|
Why wouldn't you want to use fork()?
- Switching between processes can be expensive due to context switching overhead (saving and restoring registers and possibly swapping part of the program back into memory). Also, your system's scheduler might limit the number of processes that can be active. It might even stop being effective past a certain point -- the system might spend more time figuring out which process to run than actually running the processes.
- The inherited open files and sockets could result in unexpected synchronization problems if you're not careful after calling
fork(). You'll probably want to usedup()to make private copies of file or socket descriptors -- this results in additional overhead. - Creating a copy of all a program's data might take a while. It might also waste memory that you need for running processes.
- If the parent and child, or individual children, need to communicate or share data, inter-process synchronization techniques are generally very slow.
Now that you know why you wouldn't want to use fork(), let's talk about threads and go over some code samples.
Because of fork()'s shortcomings, UNIX developers started inventing more specialized versions that would be more efficient in suitable situations. Eventually, this resulted in threads being added to the kernel as a lightweight execution stream. Processes were now defined as one or more threads of execution instead of just a stream of instructions.
When a process creates a new thread, the thread is a part of the process. All of the open files and sockets are available to the thread. More importantly, so is all of the data of the process, making communication and synchronization between threads simple and efficient (see Figure 2 and Listing 2).
Figure 2. Threads in a process
Listing 2. General threaded program layout
#include <pthread.h>
#include <stddef.h>
void *thread_function( void *data )
{
/* Do something useful. */
return NULL;
}
int main( int argc, char **argv )
{
pthread_t threadId = 0;
pthread_attr_t attr;
pthread_attr_init( &attr );
pthread_attr_setdetachstate( &attr, PTHREAD_CREATE_DETACHED );
(void)pthread_create( &threadId, &attr, &thread_function, NULL );
/* Do other things while the worker thread is running. */
return 0;
}
|
As you can see, creating a thread requires a bit more setup than just calling fork() did, but it offers a lot more flexibility and control.
Why wouldn't you want to use threads?
- You need to use thread-safe libraries or carefully orchestrate calls to functions that maintain state between calls. Traditional UNIX functions that maintain state, such as
rand(), which keeps internal data in a static variable, have thread-safe replacements,rand_r()in this case, in systems that support POSIX threads. Because thread-safe libraries are generally available, this isn't a drawback unless you forget to use the thread-safe version of a function. In which case, you'll experience odd behavior in your application and might even have a hard time debugging the problem. - Debugging thread deadlocks caused by improper use of synchronization techniques, such as mutexes and condition variables, can be painful.
- Bad programming habits, such as global variables and static variables inside functions, must be abandoned in favor of writing cleaner, modern code.
As you've seen the basic structure of a forking program and a threaded program, let's create two complete programs illustrating each approach.
In this rather contrived example, you need to generate a bunch of random numbers. You wouldn't normally go through the trouble of launching child processes or threads to do something like this, but the same approach can be applied to:
- Long I/O operations
- Network services
- Log analysis
- Cryptography
- Compression
Any task that has multiple steps can benefit from this division of labor, as long as two or more of those steps can occur at the same time.
In addition to generating (and then discarding) some random numbers, both of the programs illustrate "good form" for handling child processes and threads.
The forked approach requires a signal handler to catch when child processes exit. When it's finished forking child processes, it waits for any remaining children to exit before it returns from its own main() function.
In Listing 3, you can see the standard headers required to declare the functions and other symbols referenced by this program. During linking, you won't need to include any additional libraries, as you're using standard C library functions. You're going to use 32 separate workers; so, counting the parent, you'll have a total of 33 processes running just for this one task. Additionally, you're creating a global variable to count the number of worker processes that are still running. You'll need that at the end of the main() function in order to exit cleanly.
Listing 3. Headers and declarations
#include <errno.h> #include <signal.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #include <sys/types.h> #define FORK_WORKERS 32 volatile static int activeWorkers = 0; |
Here's the trivial work function (see Listing 4). Unless you want to follow along with the example code download, feel free to replace this with something more interesting. Note how everything is being done in this function, including generating random numbers with rand_r(). Because rand() isn't thread-safe, the code uses rand_r() instead of the standard rand(). It maintains state internally (using a static variable) -- a definite mistake for threaded code. This isn't important in a forked program, but it's still good form.
Listing 4. The worker function
void fork_worker( void )
{
unsigned int randState = 0;
int idx = 0;
randState = (unsigned int)getpid();
for( idx = 0; idx < ( FORK_WORKERS * FORK_WORKERS ); idx++ ) {
int val = rand_r( &randState );
if( val > randState ) {
val++;
} else {
val--;
}
}
exit( EXIT_SUCCESS );
}
|
This short function gets called by the SIGCHLD signal handler you're going to create down in the fork() function. The UNIX process manager sends a SIGCHLD signal to the parent process whenever one of its children exits. This gives you a chance to grab the child's exit status using the wait() function. You need to wait for the children, so they don't turn into zombie processes. Zombies are the remains of programs that have exited -- maybe they're just a process ID and an exit status, or maybe the process manager is caching additional process data. If you don't clean up zombies by using the wait() functions (wait() and waitpid(), which lets you wait for a specific process ID), your process table will fill up and you eventually won't be able to create new processes.
Signal handlers can't do much, because most functions are forbidden in a signal handler. Luckily, the wait() function is one of the few signal handler-safe functions in the standard C library, and it's generally used exactly like you're doing here.
If a process isn't waiting to have its unnatural life as a zombie process ended, the wait() function will, as the name suggests, block until a child process exits. Since the signal handler shown in Listing 5 is being called in response to a child exiting, this call to wait() is going to exit immediately.
Listing 5. The signal handler
void worker_exitted( int sig )
{
int status;
activeWorkers--;
(void)wait( &status );
}
|
After declaring some variables, Listing 6 shows you how to use sigaction() to set up the signal handler you'll need to prevent zombie processes.
The for loop is where the interesting bits are happening. When you call fork(), it splits your process into two identical processes, the original process (the parent) and the child. For the parent, it returns the process ID of the child and, in the child, it returns 0. If an error occurs while the child is being created, it returns a negative error code.
The if statement following the fork() call handles the possibility of error, sends the child process off to the fork_worker() function you looked at earlier, or counts the number of active children if it's the parent process running.
Before fork() exits, make sure that none of the child processes are still in a zombie state. At this point, the wait() calls will block if there are still child processes doing work (see Listing 6).
Listing 6. The mainline
int main( int argc, char **argv )
{
pid_t ppid = 0;
int idx = 0;
struct sigaction signalHandler;
sigset_t mask;
(void)sigemptyset( &mask );
signalHandler.sa_handler = worker_exitted;
signalHandler.sa_mask = mask;
signalHandler.sa_flags = 0;
(void)sigaction( SIGCHLD, &signalHandler, NULL );
printf( "Launching %d child processes...\n", FORK_WORKERS );
/* Create the child processes */
for( idx = 0; idx < FORK_WORKERS; idx++ ) {
ppid = fork();
if( ppid < 0 ) {
printf( "Error creating child %d: %s\n", idx,
strerror( errno ) );
} else if( ppid == 0 ) {
/* I'm a child process. */
fork_worker();
} else {
/* Parent process. */
printf( "Child %d's PID is %d\n", idx, ppid );
activeWorkers++;
}
}
sleep( 1 );
/* Wait for children to exit */
printf( "Waiting for %d children to exit...\n", activeWorkers );
while( activeWorkers > 0 ) {
int status = 0;
if( wait( &status ) == -1 ) {
/* All children have exited already. */
break;
}
activeWorkers--;
if( WIFSIGNALED( status ) ) {
printf( "Child exitted due to signal %d\n", WTERMSIG( status ) );
} else {
printf( "Child's exit status was %d\n", WEXITSTATUS( status ) );
}
}
printf( "Done!\n" );
return EXIT_SUCCESS;
}
|
The last interesting detail here is the process exit status that you get from wait() isn't the value returned by the process with a call to exit(), or any of the other methods of exiting. It's got additional data encoded in its bits, so you need to use the WIFSIGNALED() macro to get the true exit status. Note the spelling, I'm always adding an extra L in there.
Even though the threaded version looks more complex, it's actually more straightforward. The complexity of the POSIX threads API is significantly more flexible than the legacy fork() and wait() functions.
As before, you can include a few system headers and define the number of worker threads you'll launch. Note the lack of global variables.
You can also define EOK, because not all errno.h headers have a "no error" value defined (see Listing 7).
Listing 7. Headers and declarations
#include <errno.h> #include <pthread.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> #define THREAD_WORKERS 32 #if !defined(EOK) # define EOK 0 #endif |
In Listing 8, you find the thread_worker() function, which is nearly identical to the fork_worker() function you looked at earlier. You use rand_r() here so the other threads don't mess with rand()'s state while you're generating random numbers. Your system manual describes a "_r" version for any library function that keeps state between calls.
Listing 8. Worker thread
void *thread_worker( void *data )
{
unsigned int randState = 0;
int idx = 0;
randState = (unsigned int)getpid();
for( idx = 0; idx < ( THREAD_WORKERS * THREAD_WORKERS ); idx++ ) {
int val = rand_r( &randState );
if( val > randState ) {
val++;
} else {
val--;
}
}
return NULL;
}
|
The program's mainline creates the threads and then waits for them to finish their work before exiting. You don't need to make a signal handler, because threads don't live forever in a zombie state.
The call to pthread_create() creates a new thread and then starts it running. That last detail might be important if you need to do additional setup before letting the thread continue; use a mutex or some other synchronization primitive to block the new thread.
The final for loop (see Listing 9) simply goes through all of the thread IDs and uses pthread_join() to wait for each thread to finish and exit. Nothing special is required to get the thread's exit value.
Listing 9. Mainline
int main( int argc, char **argv )
{
pthread_t threadIds[THREAD_WORKERS];
int idx = 0;
int retval = 0;
printf( "Creating %d child threads...\n", THREAD_WORKERS );
/* Create and launch the child threads */
for( idx = 0; idx < THREAD_WORKERS; idx++ ) {
retval = pthread_create( &(threadIds[idx]), NULL,
&thread_worker, NULL );
printf( "Child %d's thread ID is %d\n", idx, (int)threadIds[idx] );
}
sleep( 1 );
/* Wait for children to exit */
for( idx = 0; idx < THREAD_WORKERS; idx++ ) {
int status = 0;
retval = pthread_join( threadIds[idx], (void *)&status );
if( retval == 0 ) {
printf( "Thread %d's exit status was %d\n",
(int)threadIds[idx], status );
} else {
printf( "Error joining thread %d\n",
(int)threadIds[idx] );
}
}
printf( "Done!\n" );
return EXIT_SUCCESS;
}
|
With threads, you have to be careful with shared resources, such as global variables and functions, that save state in static variables. However, by using threads, you gain a more logical way of taking advantage of multiple processors and partitioning your application's work into straightforward, lightweight execution streams.
One of the problems you'll run into while developing something like a network service on UNIX is how to handle the need to continue accepting connections while serving clients that are already connected. How can you continue to work while waiting for something else to happen?
The legacy approach is to use the standard UNIX fork() function to split your process into an identical parent and child process. The parent can go back to waiting for another connection while the child goes off and services the client. Until the child exits, the parent is interrupted with a signal. If you don't clean up your child processes properly, your process table fills up with zombies, preventing additional processes from being created.
Threaded programming is a new thing to many UNIX programmers, but it offers a much cleaner way of handling more than one thing at a time. Your process creates one or more threads to do its bidding; the mainline can go back to waiting for connections while the threads handle the clients without additional supervision. Special cleanup and signal handling code isn't necessary. Your program isn't going to unintentionally keep you from handling new connections by filling up the process table.
| Description | Name | Size | Download method |
|---|---|---|---|
| Eclipse 3.1 project demonstrating fork() | es-forkWork.zip | 23KB |
FTP
 |
HTTP
|
Download Director
|
| Eclipse 3.1 project demonstrating threads | es-threadWork.zip | 23KB |
FTP
|
HTTP
|
Download Director
|
Information about download methods
Learn
- Read "What is Eclipse, and how do I use it?" (developerWorks, November 2001) for a great start on Eclipse.
- For resources and more on working with Eclipse, visit the Get started now with Eclipse main page.
- For a detailed overview on how to use Eclipse in your C/C++ development projects, read "C/C++ development with the Eclipse Platform" (developerWorks, April 2003).
- Be sure to read this developerWork's article on
POSIX threads.
-
Take a look at the UNIX Specification (Version 2) standard for threads.
- Try this Introduction to UNIX Signals Programming.
- Want more? The developerWorks eServer™ zone hosts hundreds of informative articles and introductory, intermediate, and advanced tutorials on the eServer brand.
- The IBM developerWorks team hosts hundreds of technical briefings around the world which you can attend at no charge.
Discuss
- Participate in the eServer forums.
- developerWorks blogs: Get involved in
the developerWorks community.
Chris Herborth is an award-winning senior technical writer with more than 10 years of experience writing about operating systems and programming. When he's not playing with his son Alex or hanging out with his wife Lynette, Chris spends his spare time designing, writing, and researching (that is, playing) video games.
Comments (Undergoing maintenance)
Table of contents
Local resources
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
