Issue with UTF-/ encoding on csv file for excel

Question

EDIT:

As suggested special chars are displayed correctly if I use notepad++ to open the csv file. They are displayed correctly too when I import the csv file into

Answer 1

In python3 I just do this:

with open(file, "w+", encoding='utf-8-sig') as f:
                f.write("Vérification")

Pretty simple, right? :) You can search "utf-8-sig" in the python docs

Answer 2

I fixed this issue using UTF-8 BOM encoding.

# -*- coding: utf-8-sig-*-
import unicodecsv as csv
import codecs
import sys
reload(sys)
sys.setdefaultencoding("utf-8-sig")
def write_csv(file,headers):


    resultFile =codecs.open(file, "w+", "utf-8-sig")

    #headers=[s.encode('utf-8') for s in headers]
    wr = csv.writer(resultFile, dialect='excel',delimiter=";",encoding="utf-8-sig")
    wr.writerow(headers)

    resultFile.close()

headers=[""]
headers.append("Command")
headers.append("Vérification")
write_csv(r"C:\Users\ATHENA-HDA\AppData\Local\Temp\test2.txt",headers)

Answer 3

Python 2 solution using unicodecsv. Note that the documentation for unicodecsv says the module should be opened in binary mode (wb). Make sure to write Unicode strings. #coding is required to support non-ASCII characters in the source file. Make sure to save the source file in UTF-8.

#coding:utf8
import unicodecsv

with open('test.csv','wb') as f:
    # Manually encode a BOM, utf-8-sig didn't work with unicodecsv
    f.write(u'\ufeff'.encode('utf8'))
    w = unicodecsv.writer(f,encoding='utf8')
    # Write Unicode strings.
    w.writerow([u'English',u'Chinese'])
    w.writerow([u'American',u'美国人'])
    w.writerow([u'Chinese',u'中国人'])

Python 3 solution. #coding is optional here because it defaults to UTF-8. Just make sure to save the source file in UTF-8. unicodecsv is no longer required. The built-in csv works correctly. csv documentation says to open the file with newline=''.

#coding:utf8
import csv

with open('test.csv','w',newline='',encoding='utf-8-sig') as f:
    w = csv.writer(f)
    # Write Unicode strings.
    w.writerow([u'English',u'Chinese'])
    w.writerow([u'American',u'美国人'])
    w.writerow([u'Chinese',u'中国人'])