why should we use stdout=PIPE in subprocess.Popen?

Question

from subprocess import PIPE,Popen

p = Popen("ls -l",shell=True,stderr=PIPE,stdout=PIPE)
(out,err) = p.communicate()
print(out, err)

In above Popen call, if I remove stdout=PIPE, I am getting newline after every listing by ls -l in output. But if use stdout=PIPE, I get \n displayed, rather than newline, as below

b'total 67092\n-rw-r--r--  1 root root      171 May 27 09:08 new.py\n-rw-r--r--  1   
    root root       74 May 12 18:14 abcd.conf\n-rwxr-xr-x  1 root root     5948 May 13 13:21 abxyz.sh\ndrwxr-xr-x  2 root root     4096 May 13
12:39 log\ndrwxrwxrwx  3 root root     4096 May 14 16:02
newpy\n-rw-r--r--  1 root root      134 May 27 10:13
pipe.py\n-rw-r--r--  1 root root      155 May 27 10:07
proc.py\ndrwxrwxrwx  3 root root     4096 May 14 14:29 py\ndrwxr-xr-x
16 1000 1000\n' b''

How does PIPE exactly works in case of subprocess.Popen? Why do we need it? I got proper output without using it also? Are we using this for getting both stderr, stdout?

Answer 1

Remove the print() call to see the difference.

When you do not pipe the ls output to Python, it is instead displayed on your terminal directly; it's output goes to your terminal. If you pipe it to Python, you get to see the whole contents as bytes, including newline bytes (represented as \n).

Decode the results if you want the newlines to be printed literally:

print(out.decode('utf8'))