Efficiently assign a row to a lil_matrix

问题

How can I efficiently assign a row to a lil_matrix? I'm currently using:

Q[mid, :] = new_Q

where new_Q is the result of lil_matrix.getrow(x)

I ran a test on using Q.getrow(i) vs. Q[i, :], and found the former to be 20x faster.

Here's the lil_matrix documentation.

回答1:

These time tests on small lil (dense, but I don't think that matters), suggest that x[i,:] is not a problem setting. Yes, for some reason, it is slow when used to fetch a row.

In [108]: x=sparse.lil_matrix(np.arange(120).reshape(24,5))

In [109]: timeit x[10,:]=10
1000 loops, best of 3: 235 us per loop

In [110]: timeit y=x.getrowview(10);y[:]=10
1000 loops, best of 3: 291 us per loop

In [111]: timeit x[10,:]
1000 loops, best of 3: 658 us per loop

In [112]: timeit x.getrowview(10)
10000 loops, best of 3: 51.4 us per loop

The source for getrowview is instructive, showing how the underlying data structures for this matrix are handled.

def getrowview(self, i):
    """Returns a view of the 'i'th row (without copying).
    """
    new = lil_matrix((1, self.shape[1]), dtype=self.dtype)
    new.rows[0] = self.rows[i]
    new.data[0] = self.data[i]
    return new

I think x[10,:] uses x.__getitem__ or x.__setitem__. Both of those functions are more complicated than this getrowview. I'm guessing x.__getitem__ is slow because it is also indexing on columns (see. x._get1). x[10,:] takes just as much time as x[10,0:5].

Indicative of what might be accomplished if you only need to set one row, and access rows and data directly:

In [133]: timeit x.rows[10]=range(5);x.data[10]=[10]*5
1000000 loops, best of 3: 1.36 us per loop

This is far from general, but it gives an idea of what you can do in special cases.

Some more timings:

In [156]: timeit x[10,:]=x.getrow(12)[:,:]
1000 loops, best of 3: 940 us per loop

In [157]: timeit x[10,:]=x.getrow(12)
1000 loops, best of 3: 259 us per loop

That extra [:,:] is slow. getrow already returns a copy, so it probably is not needed.

In [160]: timeit b=x.getrowview(10);b=x.getrow(12)
10000 loops, best of 3: 104 us per loop

In [169]: timeit x.rows[10],x.data[10] = x.rows[12][:],x.data[12][:]
1000000 loops, best of 3: 1.25 us per loop

Direct modification of rows and data needs to be tested carefully.

来源：https://stackoverflow.com/questions/21817378/efficiently-assign-a-row-to-a-lil-matrix

标签

python

scipy

sparse-matrix