Python subprocess module much slower than commands (deprecated)

Question

So I wrote a script that accesses a bunch of servers using nc on the command line, and originally I was using Python\'s commands module and calling commands.getoutput() and

Answer 1

There seems to be at least two separate issues here.

First, you are improperly using Popen. Here are the problems I see:

1. Spawning multiple processes with one Popen.
2. Passing one string in as args instead of splitting args.
3. Using the shell to pass text to process rather than the builtin communicate method.
4. Using shell rather than directly spawning processes.

Here is a corrected version of your code

from subprocess import PIPE

args = ['nc', '-w', '1', 'server.com', 'port_num']
p = subprocess.Popen(args, stdin=PIPE, stdout=PIPE)
output = p.communicate("get file.ext")
print output[0]

Second, the fact that you suggest it ends faster when manually run than when run through subprocess suggests that the issue here is that you are not passing the correct string to nc. What is probably happening is that the server is waiting for a terminating string to end the connection. If you are not passing this, then the connection probably remains open until it times out.

Run nc manually, figure out what the terminating string is, then update the string passed to communicate. With these changes it should run much faster.