Developing multithreaded program debuggers
The pthread debug library (libpthdebug.a) provides a set of functions that allows developers to provide debug capabilities for applications that use the pthread library.
The pthread debug library is used to debug both 32-bit and 64-bit pthreaded applications. This library is used to debug targeted debug processes only. It can also be used to examine pthread information of its own application. This library can be used by a multithreaded debugger to debug a multithreaded application. Multithreaded debuggers are supported in the libpthreads.a library, which is threadsafe. The pthread debug library contains a 32-bit shared object and a 64-bit shared object.
Debuggers using the ptrace facility must link to the 32-bit version of the library, because the ptrace facility is not supported in 64-bit mode. Debuggers using the /proc facility can link to either the 32-bit version or the 64-bit version of this library.
The pthread debug library provides debuggers with access to pthread library information. This includes information on pthreads, pthread attributes, mutexes, mutex attributes, condition variables, condition variable attributes, read/write locks, read/write lock attributes, and information about the state of the pthread library. This library also provides help with controlling the execution of pthreads.
The pthread debug library does not report mutexes, mutex attributes, condition variables, condition variable attributes, read/write locks, and read/write lock attributes that have the pshared value of PTHREAD_PROCESS_SHARED.
Initialization
- The debugger calls the function each time the debug process stops, to see if the program that is being debugged is pthreaded.
- The debugger calls the function once and if the program that is being debugged is not pthreaded, sets a breakpoint to notify the debugger when the debug process is pthreaded.
After the debug process is pthreaded, the debugger must call the pthdb_session_init function, to initialize a session for the debug process. The pthread debug library supports one session for a single debug process. The debugger must assign a unique user identifier and pass it to pthdb_session_init which in turn will assign a unique session identifier which must be passed as the first parameter to all other pthread debug library functions, except pthdb_session_pthreaded, in return. Whenever the pthread debug library invokes a call back function, it will pass the unique debugger assigned user identifier back to the debugger. The pthdb_session_init function checks the list of call back functions provided by the debugger, and initializes the session's data structures. Also, this function sets the session flags. See the pthdb_session_setflags function in Technical Reference: Base Operating System and Extensions, Volume 1.
Call back functions
- Obtain addresses and data
- Write data
- Give storage management to the debugger
- Aid debugging of the pthread debug library
Update function
Each time the debugger is stopped, after the session has been initialized, it is necessary to call the pthdb_session_update function. This function sets or resets the lists of pthreads, pthread attributes, mutexes, mutex attributes, condition variables, condition variable attributes, read/write locks, read/write lock attributes, pthread specific keys, and active keys. It uses call back functions to manage memory for the lists.
Hold and unhold functions
- To allow a user to single step a single thread, it must be possible to hold one or more of the other threads.
- For users to continue through a subset of available threads, it must be possible to hold threads not in the set.
- The pthdb_pthread_hold function sets the hold state of a pthread to hold.
- The pthdb_pthread_unhold function sets the hold
state of a pthread to unhold. Note: The pthdb_pthread_hold and pthdb_pthread_unhold functions must always be used, whether or not a pthread has a kernel thread.
- The pthdb_pthread_holdstate function returns the hold state of the pthread.
- The pthdb_session_committed function reports the function name of the function that is called after all of the hold and unhold changes are committed. A breakpoint can be placed at this function to notify the debugger when the hold and unhold changes have been committed.
- The pthdb_session_stop_tid function informs the pthread debug library, which informs the pthread library the thread ID (TID) of the thread that stopped the debugger.
- The pthdb_session_commit_tid function returns the list of kernel threads, one kernel thread at a time, that must be continued to commit the hold and unhold changes. This function must be called repeatedly, until PTHDB_INVALID_TID is reported. If the list of kernel threads is empty, it is not necessary to continue any threads for the commit operation.
- Before the commit operation (continuing all of the tids returned by the pthdb_session_commit_tid function) is started, the debugger can call the pthdb_session_committed function to get the function name and set a breakpoint. (This method can be done once for the life of the process.)
- Before the commit operation is started, the debugger calls the pthdb_session_stop_tid function with the TID of the thread that stopped the debugger. When the commit operation is complete, the pthread library ensures that the same TID is stopped as before the commit operation.
- Use the pthdb_pthread_hold and pthdb_pthread_unhold functions to set up which pthreads will be held and which will be unheld.
- Select the method that will determine when all of the hold and unhold changes have been committed.
- Use the pthdb_session_commit_tid function to determine the list of TIDs that must be continued to commit the hold and unhold changes.
- Continue the TIDs in the previous step and the thread that stopped the debugger.
The pthdb_session_continue_tid function allows the debugger to obtain the list of kernel threads that must be continued before it proceeds with single-stepping a single pthread or continuing a group of pthreads. This function must be called repeatedly, until PTHDB_INVALID_TID is reported. If the list of kernel threads is not empty, the debugger must continue these kernel threads along with the others that it is explicitly interested in. The debugger is responsible for parking the stop thread and continuing the stop thread. The stop thread is the thread that caused the debugger to be entered.
Context functions
The pthdb_pthread_context function obtains the context information and the pthdb_pthread_setcontext function sets the context. The pthdb_pthread_context function obtains the context information of a pthread from either the kernel or the pthread data structure in the debug process's address space. If the pthread is not associated with a kernel thread, the context information saved by the pthread library is obtained. If a pthread is associated with a kernel thread, the information is obtained from the debugger using call backs. It is the debugger's responsibility to determine if the kernel thread is in kernel mode or user mode and then to provide the correct information for that mode.
When a pthread with kernel thread is in kernel mode, you cannot get the full user mode context because the kernel does not save it in one place. The getthrds function can be used to obtain part of this information, because it always saves the user mode stack. The debugger can discover this by checking the thrdsinfo64.ti_scount structure. If this is non-zero, the user mode stack is available in the thrdsinfo64.ti_ustk structure. From user mode stack, it is possible to determine the instruction address register (IAR) and the call back frames, but not the other register values. The thrdsinfo64 structure is defined in procinfo.h file.
List functions
The pthread debug library maintains lists for pthreads, pthread attributes, mutexes, mutex attributes, condition variables, condition variables attributes, read/write locks, read/write lock attributes, pthread specific keys and active keys, each represented by a type-specific handle. The pthdb_object functions return the next handle in the appropriate list, where object is one of the following: pthread, attr, mutex, mutexattr, cond, condattr, rwlock, rwlockattr or key. If the list is empty or the end of the list is reached, PTHDB_INVALID_object is reported, where object is one of the following: PTHREAD, ATTR, MUTEX, MUTEXATTR, COND, CONDATTR, RWLOCK, RWLOCKATTR or KEY.
Field Functions
Detailed information about an object can be obtained by using the appropriate object member function, pthdb_object_field, where object is one of the following: pthread, attr, mutex, mutexattr, cond, condattr, rwlock, rwlockattr or key and where field is the name of a field of the detailed information for the object.
Customizing the session
The pthdb_session_setflags function allows the debugger to change the flags that customize the session. These flags control the number of registers that are read or written to during context operations, and to control the printing of debug information.
The pthdb_session_flags function obtains the current flags for the session.
Terminating the session
At the end of the debug session, the session data structures must be deallocated, and the session data must be deleted. This is accomplished by calling the pthdb_session_destroy function, which uses a call back function to deallocate the memory. All of the memory allocated by the pthdb_session_init and pthdb_session_update functions will be deallocated.
Example of hold/unhold functions
The following pseudocode example shows how the debugger uses the hold/unhold code:
/* includes */
#include <sys/pthdebug.h>
main()
{
tid_t stop_tid; /* thread which stopped the process */
pthdb_user_t user = <unique debugger value>;
pthdb_session_t session; /* <unique library value> */
pthdb_callbacks_t callbacks = <callback functions>;
char *pthreaded_symbol=NULL;
char *committed_symbol;
int pthreaded = 0;
int pthdb_init = 0;
char *committed_symbol;
/* fork/exec or attach to the program that is being debugged */
/* the program that is being debugged uses ptrace()/ptracex() with PT_TRACE_ME */
while (/* waiting on an event */)
{
/* debugger waits on the program that is being debugged */
if (pthreaded_symbol==NULL) {
rc = pthdb_session_pthreaded(user, &callbacks, pthreaded_symbol);
if (rc == PTHDB_NOT_PTHREADED)
{
/* set breakpoint at pthreaded_symbol */
}
else
pthreaded=1;
}
if (pthreaded == 1 && pthdb_init == 0) {
rc = pthdb_session_init(user, &session, PEM_32BIT, flags, &callbacks);
if (rc)
/* handle error and exit */
pthdb_init=1;
}
rc = pthdb_session_update(session)
if ( rc != PTHDB_SUCCESS)
/* handle error and exit */
while (/* accepting debugger commands */)
{
switch (/* debugger command */)
{
...
case DB_HOLD:
/* regardless of pthread with or without kernel thread */
rc = pthdb_pthread_hold(session, pthread);
if (rc)
/* handle error and exit */
case DB_UNHOLD:
/* regardless of pthread with or without kernel thread */
rc = pthdb_pthread_unhold(session, pthread);
if (rc)
/* handle error and exit */
case DB_CONTINUE:
/* unless we have never held threads for the life */
/* of the process */
if (pthreaded)
{
/* debugger must handle list of any size */
struct pthread commit_tids;
int commit_count = 0;
/* debugger must handle list of any size */
struct pthread continue_tids;
int continue_count = 0;
rc = pthdb_session_committed(session, committed_symbol);
if (rc != PTHDB_SUCCESS)
/* handle error */
/* set break point at committed_symbol */
/* gather any tids necessary to commit hold/unhold */
/* operations */
do
{
rc = pthdb_session_commit_tid(session,
&commit_tids.th[commit_count++]);
if (rc != PTHDB_SUCCESS)
/* handle error and exit */
} while (commit_tids.th[commit_count - 1] != PTHDB_INVALID_TID);
/* set up thread which stopped the process to be */
/* parked using the stop_park function*/
if (commit_count > 0) {
rc = ptrace(PTT_CONTINUE, stop_tid, stop_park, 0,
&commit_tids);
if (rc)
/* handle error and exit */
/* wait on process to stop */
}
/* gather any tids necessary to continue */
/* interesting threads */
do
{
rc = pthdb_session_continue_tid(session,
&continue_tids.th[continue_count++]);
if (rc != PTHDB_SUCCESS)
/* handle error and exit */
} while (continue_tids.th[continue_count - 1] != PTHDB_INVALID_TID);
/* add interesting threads to continue_tids */
/* set up thread which stopped the process to be parked */
/* unless it is an interesting thread */
rc = ptrace(PTT_CONTINUE, stop_tid, stop_park, 0,
&continue_tids);
if (rc)
/* handle error and exit */
}
case DB_EXIT:
rc = pthdb_session_destroy(session);
/* other clean up code */
exit(0);
...
}
}
}
exit(0);
}