Fastest way to convert a list of indices to 2D numpy array of ones

Question

I have a list of indices

a = [
  [1,2,4],
  [0,2,3],
  [1,3,4],
  [0,2]]

What\'s the fastest way to convert this to a numpy array of ones,

Answer 1

Depending on your use case, you might look into using sparse matrices. The input matrix looks suspiciously like a Compressed Sparse Row (CSR) matrix. Perhaps something like

import numpy as np
from scipy.sparse import csr_matrix
from itertools import accumulate


def ragged2csr(inds):
    offset = len(inds[0])
    lens = [len(x) for x in inds]
    indptr = list(accumulate(lens))
    indptr = np.array([x - offset for x in indptr])
    indices = np.array([val for sublist in inds for val in sublist])
    n = indices.size
    data = np.ones(n)
    return csr_matrix((data, indices, indptr))

Again, if it fits in your use case, a sparse matrix would allow elementwise/masking operations to scale with the number of nonzeros, rather than the number of elements (rows*columns), which could bring significant speedup (for a sparse enough matrix).

Another good introduction to CSR matrices is section 3.4 of Iterative Methods. In this case, data is aa, indices is ja and indptr is ia. This format also has the benefit of being very popular among different packages/libraries.