Python wait until data is in sys.stdin

Question

my problem is the following:

My pythons script receives data via sys.stdin, but it needs to wait until new data is available on sys.stdin.

As described in th

Answer 1

I've come back to problem after a long time. The issue appears to be that Apache treats a CustomLog like a file -- something it can open, write to, close, and then reopen at a later date. This causes the receiving process to be told that it's input stream has been closed. However, that doesn't mean the processes input stream cannot be written to again, just that whichever process was writing to the input stream will not be writing to it again.

The best way to deal with this is to setup a handler and let the OS know to invoke the handler whenever input is written to standard input. Normally you should avoid heavily relying on OS signal event handling as they are relatively expensive. However, copying a megabyte of text to following only produced two SIGIO events, so it's okay in this case.

fancyecho.py

import sys
import os
import signal
import fcntl
import threading

io_event = threading.Event()

# Event handlers should generally be as compact as possible.
# Here all we do is notify the main thread that input has been received.
def handle_io(signal, frame):
    io_event.set()

# invoke handle_io on a SIGIO event
signal.signal(signal.SIGIO, handle_io)
# send io events on stdin (fd 0) to our process 
assert fcntl.fcntl(0, fcntl.F_SETOWN, os.getpid()) == 0
# tell the os to produce SIGIO events when data is written to stdin
assert fcntl.fcntl(0, fcntl.F_SETFL, os.O_ASYNC) == 0

print("pid is:", os.getpid())
while True:
    data = sys.stdin.read()
    io_event.clear()
    print("got:", repr(data))
    io_event.wait()

How you might use this toy program. Output has been cleaned up due to interleaving of input and output.

$ echo test | python3 fancyecho.py &
[1] 25487
pid is: 25487
got: 'test\n'
$ echo data > /proc/25487/fd/0
got: 'data\n'
$