Check what a running process is doing: print stack trace of an uninstrumented Python program

Question

Is there a way on Linux to check what a running Python daemon process is doing? That is, without instrumenting the code and without terminating it? Preferably I\'d like to get t

Answer 1

lptrace does exactly that. It allows you to attach to a running Python process and show currently executing functions, like strace does for system calls. You can call it like this:

vagrant@precise32:/vagrant$ sudo python lptrace -p $YOUR_PID
fileno (/usr/lib/python2.7/SocketServer.py:438)
meth (/usr/lib/python2.7/socket.py:223)

fileno (/usr/lib/python2.7/SocketServer.py:438)
meth (/usr/lib/python2.7/socket.py:223)
...

Note that it requires gdb to run, which isn't available on every server machine.

Answer 2

Some of the answers in Showing the stack trace from a running Python application are applicable in this situation:

• pyrasite (this was the one that worked for me):

  $ sudo pip install pyrasite
  $ echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
  $ sudo pyrasite 16262 dump_stacks.py # dumps stacks to stdout/stderr of the python program
  

• pyringe

• pydbattach - couldn't get this to work, but the repository https://github.com/albertz/pydbattach contains pointer to other tools
• pstack reportedly prints the python stack on Solaris

Answer 3

winpdb allows you to attach to a running python process, but to do this, you must start the python process this way:

 rpdb2 -d -r script.py

Then, after setting a password:

A password should be set to secure debugger client-server communication.
Please type a password:mypassword

you could launch winpdb to File>Attach to (or File>Detach from) the process.

Answer 4

on POSIX systems like Linux, you can use good old GDB, see

• https://t37.net/debug-a-running-python-process-without-printf.html and
• https://wiki.python.org/moin/DebuggingWithGdb

There's also the excellent PyCharm IDE (free community version available) that can attach to a running Python process right from within the IDE, using Pdb 4 under the hood, see this blog entry:

• http://blog.jetbrains.com/pycharm/2015/02/feature-spotlight-python-debugger-and-attach-to-process/

Answer 5

You can use madbg (by me). It is a python debugger that allows you to attach to a running python program and debug it in your current terminal. It is similar to pyrasite and pyringe, but newer, doesn't require gdb, and uses IPython for the debugger (which means colors and autocomplete).

To see the stack trace of a running program, you could run:

madbg attach <pid>

And in the debugger shell, enter: bt