How to do keyword mapping in pandas

问题

I have keyword

India
Japan
United States
Germany
China

Here's sample dataframe

id    Address 
1     Chome-2-8 Shibakoen, Minato, Tokyo 105-0011, Japan
2     Arcisstraße 21, 80333 München, Germany
3     Liberty Street, Manhattan, New York, United States
4     30 Shuangqing Rd, Haidian Qu, Beijing Shi, China
5     Vaishnavi Summit,80feet Road,3rd Block,Bangalore, Karnataka, India

My Goal Is make

id    Address                                                          India Japan United States  Germany China    
1     Chome-2-8 Shibakoen, Minato, Tokyo 105-0011, Japan              0     1     0              0       0                  
2     Arcisstraße 21, 80333 München, Germany                          0     0     0              1       0
3     Liberty Street, Manhattan, New York, USA                        0     0     1              0       0
4     30 Shuangqing Rd, Haidian Qu, Beijing Shi, China                0     0     0              0       1
5     Vaishnavi Summit,80feet Road,Bangalore, Karnataka, India        1     0     0              0       0

The basic idea is create keyword detector, I am thinking to use str.contain and word2vec but I can't get the logic

回答1:

Make use of pd.get_dummies():

countries = df.Address.str.extract('(India|Japan|United States|Germany|China)', expand = False)
dummies = pd.get_dummies(countries)
pd.concat([df,dummies],axis = 1)

Also, the most straightforward way is to have the countries in a list and use a for loop, say

countries = ['India','Japan','United States','Germany','China']
for c in countries:
    df[c] = df.Address.str.contains(c) * 1

but it can be slow if you have a lot of data and countries.

回答2:

In [58]: df = df.join(df.Address.str.extract(r'.*,(.*)', expand=False).str.get_dummies())

In [59]: df
Out[59]:
   id                                            Address   China   Germany   India   Japan   United States
0   1  Chome-2-8 Shibakoen, Minato, Tokyo 105-0011, J...       0         0       0       1               0
1   2             Arcisstra?e 21, 80333 Munchen, Germany       0         1       0       0               0
2   3  Liberty Street, Manhattan, New York, United St...       0         0       0       0               1
3   4   30 Shuangqing Rd, Haidian Qu, Beijing Shi, China       1         0       0       0               0
4   5  Vaishnavi Summit,80feet Road,3rd Block,Bangalo...       0         0       1       0               0

NOTE: this method will not work if country is not at the last position in Address column or if country name contains ,

回答3:

from numpy.core.defchararray import find

kw = 'India|Japan|United States|Germany|China'.split('|')
a = df.Address.values.astype(str)[:, None]

df.join(
    pd.DataFrame(
        find(a, kw) >= 0,
        df.index, kw,
        dtype=int
    )
)

   id                        Address  India  Japan  United States  Germany  China
0   1  Chome-2-8 Shibakoen, Minat...      0      1              0        0      0
1   2  Arcisstraße 21, 80333 Münc...      0      0              0        1      0
2   3  Liberty Street, Manhattan,...      0      0              1        0      0
3   4  30 Shuangqing Rd, Haidian ...      0      0              0        0      1
4   5  Vaishnavi Summit,80feet Ro...      1      0              0        0      0

来源：https://stackoverflow.com/questions/46601437/how-to-do-keyword-mapping-in-pandas

标签

python

pandas

word2vec