Python 2 CSV writer produces wrong line terminator on Windows

Question

According to the its documentation csv.writer should use '\r\n' as lineterminator by default.

import csv

with open("test.csv", "w") as f:
    writer = csv.writer(f)

    rows = [(0,1,2,3,4),
           (-0,-1,-2,-3,-4),
           ("a","b","c","d","e"),
           ("A","B","C","D","E")]           

    print writer.dialect.lineterminator.replace("\r", "\\r").replace("\n", "\\n")
    writer.writerows(rows)
    print writer.dialect.lineterminator.replace("\r", "\\r").replace("\n", "\\n")

This prints

\r\n
\r\n

as expected. But, the created csv-file uses the lineterminator '\r\r\n'

0,1,2,3,4

0,-1,-2,-3,-4

a,b,c,d,e

A,B,C,D,E

Is this a bug or is there something wrong in my usage of csv.writer?

Python version:

ActivePython 2.6.2.2 (ActiveState Software Inc.) based on Python 2.6.2 (r262:71600, Apr 21 2009, 15:05:37) [MSC v.1500 32 bit (Intel)] on win32

on Windows Vista

Answer 1

In Python 2.x, always open your file in binary mode, as documented. csv writes \r\n as you expected, but then the underlying Windows text file mechanism cuts in and changes that \n to \r\n ... total effect: \r\r\n

From the csv.writer documentation:

If csvfile is a file object, it must be opened with the 'b' flag on platforms where that makes a difference.

There seems to be some reticence about actually uttering the name of the main culprit :-)

Edit: As mentioned by @jebob in the comments to this answer and based on @Dave Burton's answer, to handle this case in both Python 2 and 3, you should do the following:

if sys.version_info >= (3,0,0):
    f = open(filename, 'w', newline='')
else:
    f = open(filename, 'wb')

Answer 2

Unfortunately, it's a bit different with the csv module for Python 3, but this code will work on both Python 2 and Python 3:

if sys.version_info >= (3,0,0):
    f = open(filename, 'w', newline='')
else:
    f = open(filename, 'wb')

Answer 3

To change the line terminator in Python 2.7 csv writer use

writer = csv.writer(f, delimiter = '|', lineterminator='\n')

This is a much simpler way to change the default delimiter from \r\n.