How to Achieve Continuous Interactive Dialog with a Subprocess in Python? [duplicate]

问题

Maybe continuous interactive isn't the right phrase. I'm wondering if someone could help me understand the basics of calling a program as a subprocess from a Python program? I've been hacking away but I keep running into frustrating errors. I understand best using simple examples. I have a program called square.py saved to my desktop that uses the following code:

i=0
while i<10:
    x=int(raw_input('Enter x-dimension: '))
    x1 = x*x
    print str(x1)
    i=i+1

Could someone please explain to me in simple terms how to call this program in IDLE and maintain continuous interactive dialogue with it (keep it open and running in the background) until it terminates on its own?

Eventually I will need to use this knowledge to call a genetic algorithm program written in C from a Python GUI (using tkinter). The genetic algorithm outputs an array of values, the user uses those values to do something and gives user feedback as to the utility of those values. The user feedback is in the form of 0-100. When the genetic algorithm receives the input it does its magic and outputs another array of numbers, which would hopefully have slightly better utility. So I want to wrap a Python GUI around a scary looking C program, feed the C program a feedback value and receive an array of numbers.

I hope I explained what I'm trying to do well enough; if anyone can help me use subprocess to call square.py, pass it a value and get back its output I would be pretty happy. Cheers!

回答1:

Programs that are designed to interact with a human are different from programs that are designed to interact with other programs. Your square.py script is closer to the former category. Possible issues:

the prompt ends in the middle of a line that forces a parent script to read one byte at a time instead of full lines or known chunks to avoid blocking
child's answers are not flushed explicitly. It means that the block buffering may be enabled in non-interactive mode e.g., when it is run via subprocess without pty

Here's how you could interact with it using subprocess module in its current form:

#!/usr/bin/env python
from __future__ import print_function
import sys
from itertools import cycle
from subprocess import Popen, PIPE
from textwrap import dedent

# start child process
p = Popen([sys.executable or 'python', '-u', '-c', dedent("""
        for i in range(10):
            x = int(input('Enter x-dimension: '))
            print(x*x)
        """)], stdin=PIPE, stdout=PIPE, universal_newlines=True, bufsize=1)
for n in cycle([3, 1, 4, 15, 926]): # infinite loop
    while p.poll() is None: # while the subprocess is running
        # send input to the child
        print(n, file=p.stdin)
        # read & parse answer
        data = p.stdout.readline().rpartition(' ')[2]
        if not data: # EOF
            answer = None
            break # exit inner loop
        answer = int(data)
        if answer == 1: # show example when input depends on output
            n += 1
        else: # done with given `n`
            break # exit inner loop
    else: # subprocess ended
        break # exit outer loop
    if answer is not None:
        print("Input %3d Output %6d" % (n, answer))
p.communicate() # close pipes, wait for the child to terminate

And here's the same thing but using pexpect (for comparison):

#!/usr/bin/env python
import sys
from itertools import cycle
from textwrap import dedent

import pexpect

child = pexpect.spawnu(sys.executable or 'python', ['-c', dedent("""
            for i in range(10):
                x = int(input('Enter x-dimension: '))
                print(x*x)
            """)])
for n in cycle([3, 1, 4, 15, 926]):
    while True:
        i = child.expect([pexpect.EOF, u'x-dimension:'])
        if i == 0: # EOF
            answer = None
            child.close()
            sys.exit()
        elif i == 1: # child waits for input
            child.sendline(str(n))
            child.expect(u'\\n\\d+\\s')
            answer = int(child.after)
            if answer == 1:
                n += 1
            else:
                break
        else:
            assert 0
    else: # child terminated
        break
    if answer is not None:
        print("Input %3d Output %6d" % (n, answer))

Both scripts are written to support Python 2 and Python 3 from the same source.

Note: there is -u argument in the subprocess-based script that allows to read the lines as soon as they are available even in non-interactive mode. pexpect-based script works without such switch. stdio-based programs can be unbuffered/make line-buffered using stdbuf, unbuffer utilities or by providing a pty.

You can see that even the simplest child script (square.py) requires to overcome several issues to even work at all.

Everything is simpler when the child program expects to be run from another program while remaining human-readable (debuggable). In this case, square.py could look like:

#!/usr/bin/env python
import sys
import time

for line in iter(sys.stdin.readline, ''): # get line as soon as it is available
    print(int(line)**2) # find square
    sys.stdout.flush()  # make the answer available immediately
    time.sleep(.5) # a delay to show that the answer is available immediately

It could be used from a subprocess-based module in "all at once" mode:

import sys
from subprocess import Popen, PIPE

L = [2, 7, 1] # numbers to be squared
p = Popen([sys.executable or 'python', 'square.py'], stdin=PIPE, stdout=PIPE,
          universal_newlines=True, bufsize=-1)
answers = map(int, p.communicate("\n".join(map(str, L)))[0].splitlines())

Or one number at a time:

#!/usr/bin/env python
import sys
from subprocess import Popen, PIPE

answers = []
p = Popen([sys.executable or 'python', 'square.py'], stdin=PIPE, stdout=PIPE,
          bufsize=1)
for c in [b'2', b'7', b'1']:
    p.stdin.write(c + b'\n')
    p.stdin.flush()
    answers.append(int(p.stdout.readline()))
    print(answers)
p.communicate() # close pipes, wait for child to finish
print(answers)

To get an array from your C program; you could json module:

import json
from subprocess import Popen, PIPE

p = Popen(['./c-program', 'other', 'args'], stdin=PIPE, stdout=PIPE, bufsize=1)
p.stdin.write(json.dumps({'parameter': 8}).encode() + b'\n') # send input
p.stdin.flush()
result = json.loads(p.stdout.readline().decode()) # e.g., {"result": [0, 0, 7]}
# ...
p.communicate() # close pipes, wait for child to finish

回答2:

"Continuous interactive" conflicts badly with the subprocess module, which uses OS-level pipes (on Unix-like systems; it necessarily does something a little different on Windows). To interact with a process the way users do on, e.g., ssh pty connections, however, you must create a pty session. Many programs assume, when talking to or via pipes, that they are not interactive.

The pexpect module is a translation of the old Don Libes expect into Python. It is aimed at this sort of idea.

The sh module also appears to have the necessary pieces to achieve the desired results.

(I have not used either one myself.)

来源：https://stackoverflow.com/questions/20185353/how-to-achieve-continuous-interactive-dialog-with-a-subprocess-in-python

标签

python

algorithm

subprocess

genetic