get output from a paramiko ssh exec_command continuously

Question

I am executing a long-running python script via ssh on a remote machine using paramiko. Works like a charm, no problems so far.

Unfortunately, the stdout (respectively the stderr) are only displayed after the script has finished! However, due to the execution time, I'd much prefer to output each new line as it is printed, not afterwards.

remote = paramiko.SSHClient()
remote.set_missing_host_key_policy(paramiko.AutoAddPolicy())
remote.connect("host", username="uname", password="pwd")

# myScript produces continuous output, that I want to capture as it appears    
stdin, stdout, stderr = remote.exec_command("python myScript.py")
stdin.close()
for line in stdout.read().splitlines():
    print(line)

How can this be achieved? Note: Of course one could pipe the output to a file and 'less' this file via another ssh session, but this is very ugly and I need a cleaner, ideally pythonic solution :)

Answer 1

As specified in the read([size]) documentation, if you don't specify a size, it reads until EOF, that makes the script wait until the command ends before returning from read() and printing any output.

Check this answers: How to loop until EOF in Python? and How to do a "While not EOF" for examples on how to exhaust the File-like object.

Answer 2

I was facing a similar issue. I was able to solve it by adding get_pty=True to paramiko:

stdin, stdout, stderr = client.exec_command("/var/mylongscript.py", get_pty=True)

Answer 3

A minimal and complete working example of how to use this answer (tested in Python 3.6.1)

# run.py
from paramiko import SSHClient

ssh = SSHClient()
ssh.load_system_host_keys()

ssh.connect('...')

print('started...')
stdin, stdout, stderr = ssh.exec_command('python -m example', get_pty=True)

for line in iter(stdout.readline, ""):
    print(line, end="")
print('finished.')

and

# example.py, at the server
import time

for x in range(10):
    print(x)
    time.sleep(2)

run on the local machine with

python -m run