How to find an inverse of a nearly singular matrix?

问题

I am realizing an algorithm using C++ and CUDA. But I got into trouble when I tried to find an inverse of a special matrix. This matrix has following features:

it is a square matrix (suppose: (m+3)x(m+3),m>0);
its transpose matrix is its self;
its main diagonal must be zeros;
it must have a 3x3 zero matrix on the bottom right corner;
you can consider this matrix in this form:H = [A ,B ;B' ,0];

I have tried some methods but all failed:

pseudo-inverse matrix:

I used matlab at first and got error or warning when I tried to use inv(H'*H): Warning: Matrix is singular to working precision or matrix is close to singular or badly scaled
some approximation methods:

the reference material is here:approximation I found two methods:Gauss-Jordan elimination and Cholesky decomposition.when I tried chol in matlab, i get following error:Matrix must be positive definite

can anybody give me some suggestions?

回答1:

It would be good to know some more information on your specific problem and, in particular, if you need the inverse per se or if you need to just invert a linear system of equations. I will try to give you directions for both the cases.

Let me start from the consideration that that your matrix is nearly singular and so your system is ill-conditioned.

DETERMINING THE INVERSE OF A NEARLY SINGULAR MATRIX

As it has been clarified in the comments and answers above, seeking the inverse of a nearly singular matrix is meaningless. What makes sense is to construct a regularized inverse of your matrix. You can do that by resorting to the spectral decomposition (Singular Value Decomposition, or SVD) of your matrix. More in detail, you can construct the singular system, remove the least significant singular values which are the source for the nearly singular behavior of the matrix, and then use the singular values and vectors to form an approximate inverse. Of course, in this case A*A_inv will only give an approximation of the identity matrix.

How can this be done on GPU? First, let me say that implementing an SVD algorithm in C++ or CUDA is by no means an easy task. There are several techniques among which you should choose depending on the accuracy you need, for example, to determine the singular values. Anyway, Matlab has a set of linear algebra functions working on GPU. Also, CULA and Magma are two libraries offering SVD calculation routines. Also, you can consider using Arrayfire which also offers linear algebra routines, including the SVD.

INVERTING A NEARLY SINGULAR SYSTEM

In this case, you should consider using some sort of Tikhonov regularization, which consists to formulating the inversion of the linear system as an optimization problem and adding a regularization term, which may depend on the features you already know about your uknowns.

For both the cases above, I recommend reading some theory. The book

M. Bertero, P. Boccacci, Introduction to Inverse Problems in Imaging

would be useful either if you have to find an approximate inverse or if you have the explicitly invert the linear system.

回答2:

The pseudo-inverse matrix is inv(H'*H)*H', since the condition number of H is very high (try cond(H)), you may need a regularization factor to obtain the pseudo-inverse matrix: inv(H'*H + lambda*eye(size(H)))*H'. The smaller the lambda, the lower bias such estimation will achieve. But too small value of lambda will lead to high variance (ill-conditioned). You may try a best-suit value.

You can of course use pinv(H) directly. The reason why pinv(H)*H ~= eye(size(H)) is because pinv(H) is just an approximation of the inverse of a matrix H with the rank lower than size(H,1). In other words, the columns in H is not completely independent.

I would like to show you a very simple example:

>>a =

     0     1     0
     0     0     1
     0     1     0

  pinv(a) * a
>>
   ans =

     0         0         0
     0    1.0000         0
     0         0    1.0000
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
>>a =  
     1     0
     0     1
     1     0

 pinv(a) * a
>>

 ans =

1.0000         0
     0    1.0000

Note a * pinv(a) is not identity matrix, because columns of a are linearly independent, not the row of a. Check this page for more details.

来源：https://stackoverflow.com/questions/20929563/how-to-find-an-inverse-of-a-nearly-singular-matrix

标签

c++

matlab

math

matrix