Pandas “.convert_objects(convert_numeric=True)” deprecated [duplicate]

Answer 1

You can effect a replacement using apply as done here. An example would be:

>>> import pandas as pd
>>> a = pd.DataFrame([{"letter":"a", "number":"1"},{"letter":"b", "number":"2"}])
>>> a.dtypes
letter    object
number    object
dtype: object
>>> b = a.apply(pd.to_numeric, errors="ignore")
>>> b.dtypes
letter    object
number     int64
dtype: object
>>> 

But it sucks in two ways:

1. You have to use apply rather than a non-native dataframe method
2. You have to copy to another dataframe--can't be done in place. So much for use with "big data."

I'm not really loving the direction pandas is going. I haven't used R data.table much, but so far it seems superior.

I think a data table with native, in-place type conversion is pretty basic for a competitive data analysis framework.

Answer 2

As explained by @EvanWright in the comments,

data['S1Q2I'] = pd.to_numeric(data['S1Q2I'])

is now the prefered way of converting types. A detailed explanation in of the change can be found in the github PR GH11133.

Answer 3

If you convert all columns to numeric at once, this code may work.

data = data.apply(pd.to_numeric, axis=0)

Answer 4

You can get it to apply correctly to a particular variable name in a dataframe without having to copy into a different dataframe like this:

>>> import pandas as pd
>>> a = pd.DataFrame([{"letter":"a", "number":"1"},{"letter":"b", "number":"2"}])
>>> a.dtypes
letter    object
number    object
dtype: object
>>> a['number'] = a['number'].apply(pd.to_numeric, errors='coerce')
>>> a.dtypes
letter    object
number     int64
dtype: object

An example based on the original question above would be something like:

data['S1Q2I'] = data['S1Q2I'].apply(pd.to_numeric, errors='coerce')

This works the same way as your original:

data['S1Q2I'] = data['S1Q2I'].convert_objects(convert_numeric=True)

in my hands, anyway....

This doesn't address the point abalter made about inferring datatypes which is a little above my head I'm afraid!

Answer 5

It depends on which version of Pandas...... if you have Pandas's version 0.18.0 this type will work ........

df['col name'] = df['col name'].apply(pd.to_numeric, errors='coerce') 

another versions ........

df['col name']=df.col name .astype(float)