How to get more detailed backtrace [duplicate]

Answer 1

If ou want a very detailled backtrace, you should use ptrace(2) to trace the process you want the backtrace.

You will be able to see all functions your process used but you need some basic asm knowledge

Answer 2

If you don't want to take the "signal a different process that runs gdb on you" approach, which I think gby is advocating, you can also slightly alter your code to call open() on a crash log file and then backtrace_symbols_fd() with the fd returned by open() - both functions are async signal safe according to the glibc manual. You'll need still -rdynamic, of course. Also, from what I've seen, you still sometimes need to run addr2line on some addresses that the backtrace*() functions won't be able to decode.

Also note fork() is not async signal safe: http://article.gmane.org/gmane.linux.man/1893/match=fork+async, at least not on Linux. Neither is syslog(), as somebody already pointed out.

Answer 3

Feed the addresses to addr2line and it will show you the file name, line number, and function name.

Answer 4

The better option I have found is libbacktrace by Ian Lance Taylor:

https://github.com/ianlancetaylor/libbacktrace

backtrace_symbols() does prints only exported symbols and could not be less portable as it requires the GNU libc.

addr2line is nice as it includes file names and line numbers. But it fails as soon as the loader performs relocations. Nowadays as ASLR is common, it will fail very often.

libunwind alone will not allow one to print file names and line numbers. To do this, one needs to parse DWARF debugging information inside the ELF binary file. This can be done using libdwarf, though. But why bother when libbacktrace gives you everything required for free?

Answer 5

Yes - pass the -rdynamic flag to the linker. It will cause the linker to put in the link tables the name of all the none static functions in your code, not just the exported ones.

The price you pay is a very slightly longer startup time of your program. For small to medium programs you wont notice it. What you get is that backtrace() is able to give you the name of all the none static functions in your back trace.

However - BEWARE: there are several gotchas you need to be aware of:

1. backtrace_symbols allocates memory from malloc. If you got into a SIGSEGV due to malloc arena corruption (quite common) you will double fault here and never see your back trace.

2. Depending on the platform this runs on (e.g. x86), the address/function name of the exact function where you crashed will be replaced in place on the stack with the return address of the signal handler. You need to get the right EIP of the crashed function from the signal handler parameters for those platforms.

3. syslog is not an async signal safe function. It might take a lock internally and if that lock is taken when the crash occurred (because you crashed in the middle of another call to syslog) you have a dead lock

If you want to learn all the gory details, check out this video of me giving a talk about it at OLS: http://free-electrons.com/pub/video/2008/ols/ols2008-gilad-ben-yossef-fault-handlers.ogg

Answer 6

If you're fine with only getting proper backtraces when running through valgrind, then this might be an option for you:

VALGRIND_PRINTF_BACKTRACE(format, ...):

It will give you the backtrace for all functions, including static ones.