How can a process intercept stdout and stderr of another process on Linux?

Question

I have some scripts that ought to have stopped running but hang around forever. Is there some way I can figure out what they\'re writing to STDOUT and STDERR in a readable

Answer 1

I'm not sure if it will work for you, but I read a page a while back describing a method that uses gdb

Answer 2

There's several new utilities that wrap up the "gdb method" and add some extra touches. The one I use now is called "reptyr" ("Re-PTY-er"). In addition to grabbing STDERR/STDOUT, it will actually change the controlling terminal of a process (even if it wasn't previously attached to a terminal).

The best use of this is to start up a screen session, and use it to reattach a running process to the terminal within screen so you can safely detach from it and come back later.

It's packaged on popular distros (Ex: 'apt-get install reptyr').

http://onethingwell.org/post/2924103615/reptyr

Answer 3

I used strace and de-coded the hex output to clear text:

PID=some_process_id
sudo strace -f -e trace=write -e verbose=none -e write=1,2 -q -p $PID -o "| grep '^ |' | cut -c11-60 | sed -e 's/ //g' | xxd -r -p"

I combined this command from other answers.

Answer 4

You can use reredirect (https://github.com/jerome-pouiller/reredirect/).

Type

reredirect -m FILE PID

and outputs (standard and error) will be written in FILE.

reredirect README also explains how to restore original state of process, how to redirect to another command or to redirect only stdout or stderr.

Answer 5

Since I'm not allowed to edit Jauco's answer, I'll give the full answer that worked for me (Russell's page relies on un-guaranteed behaviour that, if you close file descriptor 1 for STDOUT, the next creat call will open FD 1.

So, run a simple endless script like this:

import time

while True:
    print 'test'
    time.sleep(1)

Save it to test.py, run with

$ python test.py

Get the PID:

$ ps auxw | grep test.py

Now, attach gdb:

$ gdb -p (pid)

and do the fd magic:

(gdb) call creat("/tmp/stdout", 0600)
$1 = 3
(gdb) call dup2(3, 1)
$2 = 1

Now you can tail /tmp/stdout and see the output that used to go to STDOUT.

Answer 6

GDB method seems better, but you can do this with strace, too:

$ strace -p <PID> -e write=1 -s 1024 -o file

Via the man page for strace:

   -e write=set
               Perform a full hexadecimal and ASCII dump of all the
               data written to file descriptors listed in the spec-
               ified  set.  For example, to see all output activity
               on file descriptors 3 and 5 use -e write=3,5.   Note
               that  this is independent from the normal tracing of
               the write(2) system call which is controlled by  the
               option -e trace=write.

This prints out somewhat more than you need (the hexadecimal part), but you can sed that out easily.