Debug malloc tool

Edit online

Debugging applications that are mismanaging memory allocated by the malloc subsystem can be difficult and tedious. This is because there is generally no synchronicity between the insertion of an error and the exposure of its resulting symptom.

Adding to the difficulty is the inherent complexity of memory allocation, with thousands of allocations being made, undone, and accessed (perhaps) asynchronously and simultaneously, all within a multithreaded context that necessitates robust and efficient synchronization.

It is for these reasons that the focus of our debugging tools is primarily to move the time of symptom detection closer to the time of error insertion. This helps the application developer to pinpoint more precisely which section of code is responsible for committing the error.

Many different debugging tools have been developed for use with malloc. Some can be used in combination with other debugging tools and with all allocation policies; others are more limited in their use. Many of the debugging tools consume resources additional to those required by the process. It is up to the application developer to provide adequate resources when necessary.

Performance Considerations

The debug malloc tools are not appropriate for full-time, constant, or system-wide use. Although they are designed for minimal performance impact upon the application being debugged, significant negative impact on overall system throughput can result if they are used widely throughout a system. In particular, setting MALLOCDEBUG=catch_overflow in the /etc/environment file is not recommended, and will likely cause significant system problems, such as excessive use of paging space. The debug malloc tools should only be used to debug single applications or small groups of applications at the same time.

Because of the extra work involved in making various run-time checks, malloc subsystem performance will degrade variably with debug malloc tools enabled (depending on which tool is being used), but not to the point that applications will become unusable. After the problem is resolved, the debug malloc tools should be turned off to restore malloc subsystem performance.

Disk and memory considerations

With the catch_overflow or Malloc Log tools enabled, the malloc subsystem will consume significantly more memory.

For catch_overflow, each malloc request is increased by 4096 + 2 times the size of unsigned long, then rounded up to the next multiple of the PAGESIZE macro. catch_overflow might prove to be too memory-intensive to use for very large applications, but for the majority of applications that need memory debugging, the extra use of memory should not cause a problem. For large applications, the use of the debug_range and functionset options to catch_overflow can significantly lower memory usage, allowing the program to be debugged piecemeal.

For Malloc Log, an allocation record is stored for every active allocation in the process. This memory overhead can be minimized by specifying a low number of saved stack pointers.

If the application being debugged frequently calls malloc subsystem allocation routines, it might encounter memory usage problems with debug malloc tools enabled that could prevent the application from executing properly in a single segment. If this occurs, it may be helpful to enable the application to access additional memory by using the ulimit command and the -bmaxdata option of the ld command.

For the purpose of running with debug malloc tools enabled, set the ulimit for both the data (-d) and stack (-s) variables as follows: 
ulimit -d unlimited
ulimit -s unlimited

To reserve the maximum of 8 segments for a 32-bit process, the -bmaxdata option should be specified as -bmaxdata:0x80000000.

When the debug malloc tools are turned off, the default values for ulimit and -bmaxdata can be restored.

For more information about the ulimit command and the -bmaxdata option, see Large Program Support.

The debug malloc tools are not appropriate for use in some debugging situations. Because some of the debug malloc tools require the overhead of a page or more per allocation, programs that issue many small allocation requests will see their memory usage increase dramatically. These programs might encounter new failures as memory allocation requests are denied due to a lack of memory or paging space. These failures are not necessarily errors in the program being debugged, and they are not errors in the debug malloc tool.

One specific example of this is the X server, which issues numerous tiny allocation requests during its initialization and operation. Any attempt to run the X server using the X or xinit commands with catch_overflow enabled will result in the failure of the X server due to a lack of available memory. It is possible, however, to debug X in piecemeal fashion using the debug_range or functionset options. X clients in general will not encounter functional problems running with catch_overflow enabled. To use catch_overflow on an X client program, take the following steps:
  1. Start the X server with catch_overflow turned off.
  2. Start a terminal window (for example, dtterm, xterm, aixterm).
  3. Set the appropriate environment variables within the terminal window session to enable catch_overflow.
  4. Invoke the X client program to be debugged from within the same window.

Enabling debug malloc

Debug Malloc is not enabled by default, but is enabled and configured by setting the MALLOCDEBUG environment variable to the appropriate option. If more than one option is required, options can be separated by a comma (,). Options requested in tandem must be compatible with each other.

Note: To disable the debug malloc, unset the MALLOCDEBUG environment variable by using the unset MALLOCDEBUG command.

Malloc debugging tools

The following malloc debugging tools are available:

Buffer overflow detection

Memory management errors are sometimes caused by the application program writing past the end of an allocated buffer. Because this often has no immediate consequence, symptoms do not arise until much later when the memory that was overwritten (usually belonging to another allocation) is referenced and no longer contains the data originally stored therein.

The catch_overflow debug option exists to allow users to identify memory overwrites, overreads, duplicate frees, and reuse of freed memory allocated by the malloc subroutine. Memory problems detected by the catch_overflow tool result in an abort call or a segmentation violation (SIGSEGV). In most cases, when an error is detected, the application stops immediately and a core file is produced.

The catch_overflow option affects the allocations of the following allocation policies and options:
  • Default Allocation Policy
  • Watson Allocation Policy
  • Malloc Multiheap Option
  • Malloc Threadcache Option
  • Malloc Disclaim Option

The catch_overflow debug option is enabled by setting MALLOCDEBUG=catch_overflow. This will turn on identification of memory overwrites and overreads.

align

By default, the malloc subroutine returns a pointer aligned on a 2-word boundary. This is necessary for standards conformance and for programs which cannot accept unaligned memory accesses (e.g. programs using DCE components). However, due to a quirk in the implementation of the catch_overflow option, it is possible for a program to overwrite a buffer by an amount less than the alignment value without being detected by catch_overflow. The align option can be used to tell the malloc subsystem to disregard this default alignment in order to decrease or eliminate the number of bytes by which a buffer can be overwritten without detection. A custom alignment can be specified for any power of two between 0 and 4096 inclusive (e.g. 0,1,2,4,...). The values 0 and 1 are treated as the same, that is, there is no memory alignment; therefore any memory accesses beyond the allocated area will cause a SEGFAULT.

The align option is part of the catch_overflow option and is only meaningful when catch_overflow is enabled. To enable a non-default alignment, set the MALLOCDEBUG environment variable as follows: 
MALLOCDEBUG=catch_overflow,align:n
where n is the desired alignment.
To calculate how many bytes of overreads or overwrites the catch_overflow option will allow for a given allocation request when n is the requested alignment and size is the number of bytes to be allocated, use the following formula: 
((((size / n) + 1) * n) - size) % n
The following example demonstrates the effect of the align option on the application's ability to perform overreads or overwrites with the catch_overflow option enabled. In this example, the align option is specified with a value of 2: 
MALLOCDEBUG=align:2,catch_overflow
The catch_overflow option handles overreads and overwrites as follows:
  • When an even number of bytes is allocated, malloc allocates exactly the number of bytes requested, which will allow for 0 bytes of overreads or overwrites.
  • When an odd number of bytes is allocated, malloc allocates the number of bytes requested, plus one additional byte to satisfy the required alignment. This allows for 1 byte of possible overreads or overwrites.
override_signal_handling
The catch_overflow option reports errors in one of the following ways:
  • Memory access errors (such as trying to read or write past the end of allocated memory) cause a segmentation violation (SIGSEGV), resulting in a core dump.
  • For other types of errors (such as trying to free space that was already freed), the catch_overflow option will output an error message, then call the abort function , which will send a SIGIOT signal to end the current process.

If the calling program is blocking or catching the SIGSEGV and the SIGIOT signals, the catch_overflow option will be prevented from reporting errors. The override_signal_handling option provides a means of bypassing this situation without recoding and rebuilding the application.

If the override_signal_handling option is specified, the catch_overflow option will perform the following actions upon each call to a malloc subsystem routine:
  1. Disable any existing signal handlers set up by the application for SIGSEGV or SIGIOT.
  2. Set the action for both SIGIOT and SIGSEGV to the default (SIG_DFL).
  3. Unblock both SIGIOT and SIGSEGV.

If an application signal handler modifies the action for SIGSEGV between memory allocation routine calls and then attempts an invalid memory access, the catch_overflow option will be unable to report the error (the application will not exit and no core file will be produced).

Note:
  1. The override_signal_handling option can be ineffective in a threaded application environment because the catch_overflow option uses the sigprocmask subroutine and many threaded processes use the pthread_sigmask subroutine.
  2. If a thread calls the sigwait subroutine without including SIGSEGV and SIGIOT in the signal set and the catch_overflow option subsequently detects an error, the thread will hang because the catch_overflow option can only generate SIGSEGV or SIGIOT.
  3. If a pointer to invalid memory is passed to a kernel routine, the kernel routine will fail and usually return with errno set to EFAULT. If the application is not checking the return from the system call, this error might be undetected.
debug_range

By default, if the catch_overflow option is enabled, buffer overflow detection is performed for every allocation in the program. If the debug_range option is specified, only allocation requests that fall between a user-defined minimum and maximum size will have buffer overflows detected by the catch_overflow option. Otherwise, no buffer overflow detection will be performed. This option allows the user to control the amount of extra memory resources consumed by the catch_overflow option by only using the tool in specific cases.

The debug_range option is only meaningful in the context of the catch_overflow option. It is enabled as follows: 
MALLOCDEBUG=catch_overflow,debug_range:min:max
where min is the lower bound and max is the upper bound of the range in which buffer overflow detection is to be performed. If 0 is specified as a minimum value, then anything that is less than the maximum value will have buffer overflow detection performed. If 0 is specified as a maximum value, then anything that is greater than the minimum value will have buffer overflow detection performed.

Limitation: Due to an internal implementation requirement, each allocation will still necessarily be at least a page size in length. Therefore the debug_range option merely reduces the overhead of the catch_overflow option rather than eliminating it.

If the realloc subroutine is called with an allocation request that falls within the user-specified range , buffer overflow detection is performed even if the original allocation was not within the specified range. The reverse of this is also true.

Note: If the override_signal option is set in conjunction with the debug_range option, the overriding of the SIGIOT and SIGSEGV signal behavior is performed for all allocations.
functionset

Due to an internal implementation requirement, each allocation will still necessarily be at least a page size in length. Therefore the functionset option merely reduces the overhead of the catch_overflow option rather than eliminating it.

If the realloc subroutine is called from a function that is a member of the user-specified function list , buffer overflow detection is performed even if the original allocation was not made from a specified function. The reverse of this is also true.

Note: If the override_signal option is set in conjunction with the functionset option, the overriding of the SIGIOT and SIGSEGV signal behavior is performed for all allocations.

The functionset option does not check the validity of the functions specified in the list.

allow_overreading

By default, when the catch_overflow debug option is enabled and the calling program attempts to read past the end of allocated memory, a segmentation violation will occur and the process will core dump. However, the user may not be interested in catching this type of error, and may have enabled catch_overflow in order to catch more dangerous overwrites. Specifying the allow_overreading option will cause the catch_overflow option to ignore overreads so that other types of errors, which may be considered more serious, can be detected first.

The allow_overreading option is only meaningful in the context of the catch_overflow option. It is enabled as follows: 
MALLOCDEBUG=catch_overflow,allow_overreading,
postfree_checking

The postfree_checking option consumes a substantial amount of extra memory. Programs with very large memory requirements may not be able to use the postfree_checking option.

Malloc trace

Malloc Trace is a debugging option designed to allow tracing of all calls to the malloc subsystem API through the system trace facility.

Mallow log

Malloc Log is a debugging option designed to provide the user with a runtime database of active allocations in the malloc subsystem.

report_allocations

The report_allocations option is a tool for detecting memory leaks in an application program. The report_allocations option uses the database constructed by Malloc Log to report a list of allocations currently held by the user. A record of each successful allocation is made at the time of the request by Malloc Log. When an allocation is deallocated, Malloc Log removes its record from the database. At process exit, the list of allocations still active is printed to stderr, giving a list of allocations that were never freed by their callers.

The report_allocations option requires the functionality of Malloc Log to work. Thus, Malloc Log is implicitly enabled when report_allocations is enabled. The report_allocations option is enabled as follows: 
MALLOCDEBUG=report_allocations
validate_ptrs

By default, the malloc subsystem APIs do not validate their input pointers to ensure that they actually reference memory previously allocated. If one of these pointers is invalid, severe heap corruption can occur. Specifying the validate_ptrs option causes the malloc subsystem APIs to perform extensive validation on their input pointers. If a pointer is found to be invalid (that is, it does not reference memory previously allocated by a call to the malloc subsystem API), an error message stating why it is invalid is printed, the abort function is called, and a core file is produced. The validate_ptrs option is similar to the verbose sub-option. The validate_ptrs option does not take effect if the postfree_checking option is enabled.

The validate_ptrs option is enabled as follows: 
MALLOCDEBUG=validate_ptrs
Malloc detect

Malloc Detect is a debugging option designed to detect and report corruption of the internal malloc subsystem data structures on every call to a malloc subsystem API.

verbose

Sub-option of Malloc Detect.

checkarena

Sub-option of Malloc Detect.

output

By default, the malloc debugging options send their output to stderr. This may not be desired for all programs. The output option exists to provide an alternate destination for printed information. Output can be sent to either stderr, stdout, or to any file on the system.

The output option is enabled as follows: 
MALLOCDEBUG=output:<filename>
continue

Many malloc debugging options call abort() when they detect an error. This is not always the desired behavior for all programs. The continue option exists to instruct the malloc subsystem to continue after the detection of a synchronous error rather than to abort the process. Error messages will still be logged to the appropriate channels.

The continue option is enabled as follows: 
MALLOCDEBUG=continue
Malloc debug fill

Malloc debug fill is a debugging option designed to fill up the memory allocated through the malloc() calls with user specified pattern for debug purposes..

The pattern should be specified as a string (for example, export MALLOCDEBUG=fill:”abc” will set the memory allocated through malloc with the pattern “abc”) and a maximum of 128 characters is allowed. If the pattern is not specified, fill option is ignored.

The malloc debug fill option can be enabled as follows:

MALLOCDEBUG=fill:pattern

Pattern can be a octal or hexadecimal numbers specified in the form of a string. i.e the pattern “\101” , is treated as the octal notation for character ‘A’ and the pattern “\x41”, is treated as the hexadecimal notation for character ‘A’

If an invalid octal number is specified, for example \777 which cannot be contained within 1 byte, will be stored as \377, the maximum octal value that can be stored as 1 byte.