Matrix search operation using numpy and pandas

问题

I am trying to search from one matrix and replace that value on 2nd matrix.

ds1 = [[ 4, 13,  6,  9],
       [ 7, 12,  5,  7],
       [ 7,  0,  4, 22],
       [ 9,  8, 12,  0]]
ds2 = [[ 4,  1],
       [ 5,  3],
       [ 6,  1],
       [ 7,  2],
       [ 8,  2],
       [ 9,  3],
       [12,  1],
       [13,  2],
       [22,  3]]
output = [[1, 2, 1, 3],
       [2, 1, 3, 2],
       [2, 0, 1, 3],
       [3, 2, 1, 0]]

Here is the code:

out = ds1.copy()
_,C = np.where(ds1.ravel()[:,None] == ds2[:,0])
newvals = ds2[C,1]
valid = np.in1d(ds1.ravel(),ds2[:,0])
out.ravel()[valid] = newvals

output is the result of replacing ds2 key value by it's index val in ds1. Same thing I did with my actual matrix values

ds1 = pd.read_table('https://gist.githubusercontent.com/karimkhanp/9527bad750fbe75e072c/raw/ds1', sep=' ', header=None)
ds2 = pd.read_table('https://gist.githubusercontent.com/karimkhanp/1692f1f76718c35e939f/raw/6f6b348ab0879b702e1c3c5e362e9d2062e9e9bc/ds2', header=None, sep=' ')

so I get

   _,C = np.where(ds1.ravel()[:,None] == ds2[:,0])
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/generic.py", line 1947, in __getattr__
    (type(self).__name__, name))
AttributeError: 'DataFrame' object has no attribute 'ravel'

I also tried by converting in numpy array

ds1 = np.array(ds1)
ds2 = np.array(ds2)
_,C = np.where(ds1.values.ravel()[:,None] == ds2.values[:,0])

so it gave:

AttributeError Traceback (most recent call last)<ipython-input-39-6a80d7cd7f81> in <module>()----> 1 _,C = np.where(ds1.values.ravel()[:,None] == ds2.values[:,0])AttributeError: 'numpy.ndarray' object has no attribute 'values'

Any suggestion or help much appreciated

回答1:

values is a member of pandas DataFrame instead of numpy ndarray. Thus, in your second method, don't convert ds to numpy array. Just remove these two lines ds1 = np.array(ds1) ds2 = np.array(ds2) and _,C = np.where(ds1.values.ravel()[:,None] == ds2.values[:,0]) should work.

----------------- This is a test on my machine -------------------

my script is

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import sys
import pandas as pd
import numpy as np

ds1 = pd.read_table('https://gist.githubusercontent.com/karimkhanp/9527bad750fbe75e072c/raw/ds1', sep=' ', header=None)
ds2 = pd.read_table('https://gist.githubusercontent.com/karimkhanp/1692f1f76718c35e939f/raw/6f6b348ab0879b702e1c3c5e362e9d2062e9e9bc/ds2', header=None, sep=' ')

print ds1.shape, ds2.shape
_,C = np.where(ds1.values.ravel()[:,None] == ds2.values[:,0])
print C

and the output is

(1000, 1001) (4000, 2)
[  10   35   60 ..., 3869 3938 3987]

My environment is cygwin and python 2.7.9.

来源：https://stackoverflow.com/questions/30293881/matrix-search-operation-using-numpy-and-pandas

标签

python

numpy

matrix

pandas