Sparse Matrix Assignment becomes very slow in Matlab

Question

I am filling a sparse matrix P (230k,290k) with values coming from a text file which I read line by line, here is the (simplified) code

while ...
            C = textscan(text_line,'%d','delimiter',',','EmptyValue', 0);
            line_number = line_number+1;
            P(line_number,:)=C{1};
end

the problem I have is that while at the beginning the

P(line_number,:)=C{1};

statement is fast, after a few thousands lines become exterely slow, I guess because Matlab need to find the memory space to allocate every time. Is there a way to pre-allocate memory with sparse matrixes? I don't think so but maybe I am missing something. Any other advise which can speed up the operation (e.g. having a lot of free RAM can make the difference?)

Answer 1

By far the fastest way to generate a sparse matrix wihtin matlab is to load all the values in at once, then generate the sparse matrix in one call to sparse. You have to load the data and arrange it into vectors defining the row and column indices and values for each filled cell. You can then call sparse using the S = sparse(i,j,s,m,n) syntax.

Answer 2

There's a sixth input argument to sparse that tells the number of nonzero elements in the matrix. That's used by Matlab to preallocate:

S = sparse(i,j,s,m,n,nzmax) uses vectors i, j, and s to generate an m-by-n sparse matrix such that S(i(k),j(k)) = s(k), with space allocated for nzmax nonzeros.

So you could initiallize with

P = sparse([],[],[],230e3,290e3,nzmax);

You can make a guess about the number of nonzeros (perhaps checking file size?) and use that as nzmax. If it turns you need more nonzero elements in the end, Matlab will preallocate on the fly (slowly).