Read a file from server with SSH using Python

Question

I am trying to read a file from a server using SSH from Python. I am using Paramiko to connect. I can connect to the server and run a command like cat filename

Answer 1

What do you mean by "line by line" - there are lots of data buffers between network hosts, and none of them are line-oriented.

So you can read a bunch of data, then split it into lines at the near end.

ssh otherhost cat somefile | python process_standard_input.py | do_process_locally

Or you can have a process read a bunch of data at the far end, break it up, and format it line by line and send it to you.

scp process_standard_input.py otherhost
ssh otherhost python process_standard_input.py somefile |  do_process_locally

The only difference I would care about is what way reduces the volume of data over a limited network pipe. In your situation it may, or may not matter.

There is nothing wrong in general with using cat over an SSH pipe to move gigabytes of data.