Python blockless subproccess input with constant output on Windows

Question

I am trying to run a command with subproccess and the _thread modules. The subproccess has a stream of output. To combat this I used two threads, one constantly prints new l

Answer 1

proc.communicate() waits for the subprocess to finish therefore it can be used at most once – you can pass all input at once and get all the output after the child process exits.

If you are not modifying input/output then you do not need to redirect subprocess' stdin/stdout.

To feed input to a subprocess in a background thread and to print its output as soon as it arrives line-by-line:

#!/usr/bin/env python3
import errno
from io import TextIOWrapper
from subprocess import Popen, PIPE
from threading import Thread

def feed(pipe):
    while True:
        try: # get input
            line = input('Enter input for minecraft')
        except EOFError:
            break # no more input
        else:
            # ... do something with `line` here

            # feed input to pipe
            try:
                print(line, file=pipe)
            except BrokenPipeError:
                break # can't write to pipe anymore
            except OSError as e:
                if e.errno == errno.EINVAL:
                    break  # same as EPIPE on Windows
                else:
                    raise # allow the error to propagate

    try:
        pipe.close() # inform subprocess -- no more input
    except OSError:
        pass # ignore

with Popen(["java", "-jar", "minecraft_server.jar"],
           cwd=r'C:\Users\Derek\Desktop\server',
           stdin=PIPE, stdout=PIPE, bufsize=1) as p, \
     TextIOWrapper(p.stdin, encoding='utf-8', 
                   write_through=True, line_buffering=True) as text_input:
    Thread(target=feed, args=[text_input], daemon=True).start()
    for line in TextIOWrapper(p.stdout, encoding='utf-8'):
        # ... do something with `line` here
        print(line, end='')

Note about p.stdin:

1. print() adds a newline at the end of each line. It is necessary because input() strips the newline
2. p.stdin.flush() is called after each line (line_buffering=True)

The output from minecraft may be delayed until its stdout buffer is flushed.

If you have nothing to add around the "do something with line here" comments then do not redirect corresponding pipes (ignoring character encoding issues for a moment).

TextIOWrapper uses the universal newline mode by default. Specify newline parameter explicitly if you do not want that.