Only first subprocess.Popen(…, stdin=f) in a loop works correctly

问题

My main goal here is to get all the cpu charge of a list of Linux-connected computers. I have been struggling and searching on the net for some time but I must miss something as I can't find my answer. So I defined a cpu_script.py :

import psutil

print(psutil.cpu_percent(interval=1,percpu=True))

to be called in my main script, which is in the same folder, with:

import subprocess
import os
import numpy as np
import psutil

usr = "AA"
computer = ["c1", "c2", "c3"] #list of computer which cpu load is to be tested
cpu_script = os.path.join(os.getcwd(),"cpu_script.py")

with open(cpu_script,"rb") as f:
    for c in computer:
        input(c)
        process = subprocess.Popen(["ssh","-X",usr + "@" + c,"python3","-u","-"], stdin=f, stdout=subprocess.PIPE)
        out = process.communicate()[0]
        input(out)

Now here is what I got from these input :

>> c1 #first computer
>> <subprocess.Popen object at 0x7fd210aab358>
>> b'[1.0, 7.1, 0.0, 1.0, 2.0, 2.0, 3.0, 0.0, 0.0, 0.0, 0.0, 1.0]\n'
>> c2 #second computer
>> <subprocess.Popen object at 0x7fd210aab390>
>> b''
>> c3 #third computer
>> <subprocess.Popen object at 0x7fd210aab390>
>> b''

So here is my question : why are the second and third output empty ? How could I get them ?

I suspected that my first process was not well "closed", so I tried to add a process.wait() or a process.kill() just after the input(out), but to no avail.

Thank you in advance for your help !

edit: subprocess.check_output() gives the same outputs. I also tried subprocess.run with:

with open(cpu_script,"rb") as f:
    for c in computer:
        input(c)
        process = subprocess.run(["ssh","-X",usr + "@" + c,"python3","-u","-"], stdin=f, stdout=subprocess.PIPE)
        input(out)

and got:

>> c1 #first computer
>> CompletedProcess(args=['ssh', '-X', 'usr@c1', 'python3', '-u', '-'], returncode=0, stdout=b'[2.0, 1.0, 1.0, 2.9, 7.0, 0.0, 2.0, 1.0, 0.0, 0.0, 0.0, 1.0]\n')
>> c2 #second computer
>> CompletedProcess(args=['ssh', '-X', 'usr@c2', 'python3', '-u', '-'], returncode=0, stdout=b'')
>> c3 #third computer
>> CompletedProcess(args=['ssh', '-X', 'usr@c3', 'python3', '-u', '-'], returncode=0, stdout=b'')

回答1:

The problem here is that once your file has been read once, the pointer is at the end of the file, so there's nothing left to read (so the second time you pass stdin=f for the same file, what's left is just empty).

Invert your inner and outer loops to reopen the file once every time you want to use it:

for c in computer:
    with open(cpu_script, "rb") as f:
        process = subprocess.Popen(["ssh", "-X", "-l", usr, c, "python3 -u -"],
                                   stdin=f, stdout=subprocess.PIPE)
        out = process.communicate()[0]

...or rewind back to the beginning between inner loops using the seek() function:

with open(cpu_script, "rb") as f:
    for c in computer:
        f.seek(0)   ### <- THIS RIGHT HERE
        process = subprocess.Popen(["ssh", "-X", "-l", usr, c, "python3 -u -"],
                                   stdin=f, stdout=subprocess.PIPE)
        out = process.communicate()[0]

来源：https://stackoverflow.com/questions/50866386/only-first-subprocess-popen-stdin-f-in-a-loop-works-correctly

标签

python

python-3.x

subprocess

psutil