OSError: [Errno 22] Invalid argument in subprocess

Question

Python 3.3.3 Windows 7

Here is the full stack:
Traceback (most recent call last):
  File \"Blah\\MyScript.py\", line 578, in Call
    output = process.commun         


        

Answer 1

@eryksun very well analyzed the core of the issue, but I think the bigger picture (the obvious things, probably) is missing in his answer.

So, anyone has an idea as to what is happening?

Your code suffers from a race condition. Sometimes your child process does not behave the way you think it behaves.

OSError: [Errno 22] Invalid argument is raised in your case when the child exits before communicate attempts to write to the named pipe, which subprocess has established between your parent and stdin of your child process.

Popen() does a lot under the hood. In your case, it first creates three named pipes (in _get_handles()), via _winapi.CreatePipe(), one for each of stdin/out/err. Then, it spawns the child process (in _execute_child()), using _winapi.CreateProcess().

_execute_child() finishes with cleanup procedures. Remember: all of this happens within Popen().

Only after Popen() returns, your Python VM in the parent process is about to proceed with the invocation of output = process.communicate(input=SPACE_KEY, timeout=600)

Given that you are on a multi core system, your system has time slices available to let the child process do some work while your Python interpreter is still within execution of Popen().

That is, there is a narrow time window between _winapi.CreateProcess() (after which the child process does some work) and Python's attempt to write to the child's stdin via communicate() (which makes a call to Windows' WriteFile, as eryksun nicely explained).

When your child exits in that time window you retrieve named error.

Why your child process exits earlier than expected? Only you can tell. Obviously, it does not always wait for data coming from stdin.