How do you read a segfault kernel log message

Question

This can be a very simple question, I\'m am attempting to debug an application which generates the following segfault error in the kern.log

kernel

Answer 1

If it's a shared library

You're hosed, unfortunately; it's not possible to know where the libraries were placed in memory by the dynamic linker after-the-fact.

Well, there is still a possibility to retrieve the information, not from the binary, but from the object. But you need the base address of the object. And this information still is within the coredump, in the link_map structure.

So first you want to import the struct link_map into GDB. So lets compile a program with it with debug symbol and add it to the GDB.

link.c

#include 
toto(){struct link_map * s = 0x400;}

get_baseaddr_from_coredump.sh

#!/bin/bash

BINARY=$(which myapplication)

IsBinPIE ()
{
    readelf -h $1|grep 'Type' |grep "EXEC">/dev/null || return 0
    return 1
}

Hex2Decimal ()
{
    export number="`echo "$1" | sed -e 's:^0[xX]::' | tr '[a-f]' '[A-F]'`"
    export number=`echo "ibase=16; $number" | bc`
}

GetBinaryLength ()
{
    if [ $# != 1 ]; then
    echo "Error, no argument provided"
    fi
    IsBinPIE $1 || (echo "ET_EXEC file, need a base_address"; exit 0)
    export totalsize=0
    # Get PT_LOAD's size segment out of Program Header Table (ELF format)
    export sizes="$(readelf -l $1 |grep LOAD |awk '{print $6}'|tr '\n' ' ')"
    for size in $sizes
    do Hex2Decimal "$size"; export totalsize=$(expr $number + $totalsize); export totalsize=$(expr $number + $totalsize)
    done
    return $totalsize
}

if [ $# = 1 ]; then
    echo "Using binary $1"
    IsBinPIE $1 && (echo "NOT ET_EXEC, need a base_address..."; exit 0)
    BINARY=$1
fi

gcc -g3 -fPIC -shared link.c -o link.so

GOTADDR=$(readelf -S $BINARY|grep -E '\.got.plt[ \t]'|awk '{print $4}')

echo "First do the following command :"
echo file $BINARY
echo add-symbol-file ./link.so 0x0
read
echo "Now copy/paste the following into your gdb session with attached coredump"
cat <l_addr)
printf "add-symbol-file .%s %#.08x\n", \$mylinkmap->l_name, \$mylinkmap->l_addr
end
set \$mylinkmap = \$mylinkmap->l_next
end

it will print you the whole link_map content, within a set of GDB command.

It itself it might seems unnesseray but with the base_addr of the shared object we are about, you might get some more information out of an address by debuging directly the involved shared object in another GDB instance. Keep the first gdb to have an idee of the symbol.

NOTE : the script is rather incomplete i suspect you may add to the second parameter of add-symbol-file printed the sum with this value :

readelf -S $SO_PATH|grep -E '\.text[ \t]'|awk '{print $5}'

where $SO_PATH is the first argument of the add-symbol-file

Hope it helps