Pandas - Writing an excel file containing unicode - IllegalCharacterError

Question

I have the following code:

import pandas as pd

x = [u\'string with some unicode: \\x16\']
df = pd.DataFrame(x)

If I try to write this datafram

Answer 1

When I encounter this error, I usually go around it by writing the file to a '.csv instead of '.xlsx' files. So instead of

yourdataframe.to_excel('Your workbook name.xlsx')

I would do:

yourdataframe.to_csv('Your workbook name.csv')

It appears the way pandas decodes .csv files by default is:

encoding : string, optional
A string representing the encoding to use in the output file,
defaults to 'ascii' on Python 2 and 'utf-8' on Python 3.

On the other hand default encoding of .xlsx files is:

encoding: string, default None
encoding of the resulting excel file. Only necessary for xlwt,
other writers support unicode natively.

This difference is responsible for that error. You will also get the error when you write data with strings that start with - or + to a .xlsx file.