== DEBUGGING CORE FILES [04] == STACK TRACE

Body

== DEBUGGING CORE FILES [04] == STACK TRACE

Now we know how to package, ship and load a core file. Question now is
what can we do with it? Most likely anyone who is a bit familiar with
C programming heard about 'stack'. But what is it really and how can
it help when it comes to core file analysis?

The 'stack' is a memory region dedicated to functions. It will provide
storage for function's local variables as well as storage for special
variables that will be used to guarantee flawless calls and returns
between various functions during execution.

One of the first thing one does after loading a core file in the debugger
is to look at the 'stack trace'. The stack trace provides the exact
location of the process at the time the core file was generated. This is
usually the starting point as it indicates where to start looking in the
source code of the program to find out what the problem is (if any).


- How to read a stack trace? -

  The schema is the same but a few things might vary depending on the
  platform, the compilation mode (-g or not...) and other things.
  Some information might appear for some platforms but not for others.
  Usually each line is made of:

    [call or return address] [function name] [args] [offset] [object name]

  call address  : The address of the instruction in that function where the
                  call to the 'callee' was made.

  return address: Address at which the execution will resume once the
                  'callee' returns.

  function name : Name of the function, mangled or not if in C++ module.

  args          : Value of the arguments passed to the function. Note that
                  in the case of a non -g program those values are not reliable.

  offset        : The offset of the instruction in the function assembly code.

  object name   : The name of the object that contains the function code.

  Now let's see one example. This is on LINUX X86.

    #0  0x00002aaab74a5b1a in semtimedop () from libc.so.6
    #1  0x00002aaab205a5a4 in sqloWaitEDUWaitPost () from libdb2e.so.1
    #2  0x00002aaaae4cd896 in sqmFastWriter::executeFastWriter() () from libdb2e.so.1
    #3  0x00002aaaae04f542 in sqleIndCoordProcessRequest(sqeAgent*) () from libdb2e.so.1
    #4  0x00002aaaae05f5e9 in sqeAgent::RunEDU() () from libdb2e.so.1
    #5  0x00002aaaaf879227 in sqzEDUObj::EDUDriver() () from libdb2e.so.1
    #6  0x00002aaaaf0318c3 in sqloEDUEntry () from libdb2e.so.1
    #7  0x00002aaaaacd6dc5 in start_thread () from libpthread.so.0
    #8  0x00002aaab74a428d in clone () from libc.so.6

    sqloWaitEDUWaitPost ():
    This is the function name, demangled here, where the call to 'semtimedop()'
    was performed from.

    libdb2e.so.1:
    This is the object in which function 'sqeAgent::RunEDU()' is defined.

    0x00002aaaaf0318c3:
    This is the address at which the execution will resume once the callee,
    'sqzEDUObj::EDUDriver()' here, will return after terminating. Note that
    the current function 'semtimedop()' that appears as the last one on
    the stack trace did not call any function so in that specific case the
    address '0x00002aaab74a5b1a' is the address of the current instruction
    executing when the core file was generated. On some platforms the address
    would actually be the address at which the 'call' instruction was made.
    In that case the execution would return to the next instruction.

  Stack traces can be provided by a debugger from either a core file or
  a running process as well as other utilities like procstack, gstack,
  pstack and so on depending on the platform you are using. But whatever
  way it is produced it is always the location in the execution path
  of the process at the time that stack trace was produced.