Improving a badly conditioned matrix

Question

I have a badly conditioned matrix, whose rcond() is close to zero, and therefore, the inverse of that matrix does not come out to be correct. I have tried using pinv() but t

Answer 1

As already pointed out in the comments, the answer to your question crucially depends on your application. Maybe adding a small multiple of the identity matrix is the right thing to do, maybe not. In order to determine that you need to tell us: How did this matrix arise? And what do you need the inverse for?

Two common cases are:

• If you know the matrix A exactly, e.g. because it is the design matrix in a general linear model b = A * X, then modifying it is not a good idea. In this case, the matrix defines a linear system of equations, and if the matrix is singular this means there is no unique solution to this system. To pick one from the infinitely many possible solutions, there are different strategies: X = A \ b picks a solution with as many zero coefficients as possible, while X = pinv(A) * b picks the solution with the minimum L2 norm. See the examples in the documentation of pinv.

• If the matrix A is estimated from data, e.g. a covariance matrix for an LDA classifier, and you have reason to believe the true value is not singular and the singularity is just due to not having enough data points for the estimate, then applying regularization or "shrinkage" by adding a small multiple of the identity matrix is a common strategy. In this case, Schäfer and Strimmer (2005) describe a way to estimate the optimal regularization coefficient from the data itself.

But I'm sure there are other cases with other answers, too.