问题
My problem is this: I'm attempting a spectral decomposition of a random process via a (truncated) Karhunen-Loeve transform, but my covariance matrix is actually a 1-parameter family of matrices, and I need a way to estimate / visualize how my random process depends on this parameter. To do this, I need a way to track the eigenvectors produced by numpy.linalg.eigh().
To give you an idea of my issue, here's a sample toy problem: Suppose I have a set of points {xs}, and a random process R with covariance C(x,y) = 1/(1+a*(x-y)^2) which depends on the parameter a. For a random sample of grid points in the range [0,1] and a given choice of a (say, a=1), I can populate my covariance matrix and implement the Karhunen-Loeve transform using:
num_x = 1000
xs = numpy.array([numpy.random.uniform() for i in range(num_x)])
z=numpy.random.standard_normal(num_x)
a0=1
def cov(x,y,a=a0): return 1/(1+a*(x-y)^2)
cov_m = numpy.array(map(lambda y: map(lambda x: cov(x,y),xs),xs))
w,v=numpy.linalg.eigh(cov_m)
R=numpy.dot(v,z*w^0.5)
This will give me realizations of R with values defined at each of my random grid points xs. What I need to be able to do, however, is - for a specific realization (which means a specific choice of my grid xs and my random coefficients z) - track how R varies with respect to the parameter a in my covariance function.
This would be easy to do if I could compute the covariance matrix symbolically and specify a after the fact. However, for large matrices this is not a plausible option. The alternative method is to find a way to keep track of each eigenvector returned by numpy.linalg.eigh(). Unfortunately, numpy appears to be reordering them so as to always list the smallest eigenvalue first; this means that, when I vary a, the eigenvectors get reordered unpredictably, and the dot product numpy.dot(v,z*w^0.5) is no longer assigning the same coefficient to the same eigenvector.
Is there a way around this?
(This is a cross-post from ASKSAGE. My implementation is using Sage, but as the question is not Sage specific and as there seems to be more activity here I thought I'd repost. My apologies if cross-posting is not acceptable; if so please delete this.)
EDIT: Based on the conversation below, I see I need to add more detail about the nature of this problem.
The idea behind a Karhunen-Loeve transform is to decompose a random process R(x) spectrally, as follows:
R(x) = \sum_{i} Z_i \lambda_i^{1/2} \phi^{(i)}(x),
where each Z_i is an i.i.d. random variable with the standard normal distribution, each \phi^{(i)}(x) is a non-random function of x determined by the solution of the eigenvectors of the covariance matrix, and each \lambda_i is the corresponding eigenvalue associated with \phi^{(i)}.
To demonstrate dependence of R on the parameter a, I need to be able to uniquely assign each \phi^{(i)} a coefficient Z_i. It doesn't matter how I do this assignation, since all of the Z's are identically distributed, but the assignation needs to be unique and it cannot depend on the corresponding value of \lambda_i, since that will depend on the parameter a.
The analytic solution is easy: just compute the eigenvectors and eigenvalues for arbitrary a, choose a specific realization of R by specifying my Z's, and watch how that realization of R depends on a. (For the toy model, R will generally become more rapidly varying as a increases.)
The problem here is how to implement this numerically. Numpy apparently scrambles the eigenvectors as it does its computation, so there's no way to uniquely tag them. I'm guessing that the only way to do this is to dig into the underlying code and specifically implement some type of arbitrary tagging function.
To put the problem succinctly: I need a way of ordering the eigenvectors produced by numpy which doesn't depend upon the magnitude of the associated eigenvalue.
Is there a way to do this?
Update: I've managed to find a partial answer to this problem, but I'll need to do some more research for the full answer. It looks like I'm going to have to implement my own algorithm for this.
For matrices of the type I'm considering, the Lanczos algorithm (for a given choice of initial vector) is deterministic and the steps do not depend on choice of my parameter. That gives me a symmetric, tridiagonal matrix to solve eigenvalues for.
Divide-and-conquer might work here. It seems like I could implement a version of it which would allow me to track the eigenvectors independent of the associated eigenvalue. The "divide" part, at least, can be implemented in a deterministic, parameter-independent way, but I'll need to learn much more about the "conquer" part of the algorithm to know for sure.
回答1:
After a bit of research, I've managed to come up with two partial answers to this problem.
The first is that, for a real symmetric matrix with no zero eigenvectors (it might also be necessary to specify non-degenerate), it should be feasible to generate an algorithm for solving the eigenpair problem which generates eigenpairs in a fixed order independent of the choice of matrix. Given a constant starting vector, the Lanczos algorithm will produce a tridiagonal matrix for an arbitrary real, symmetric matrix in a deterministic way. The "divide" part of the divide-and-conquer algorithm is similarly deterministic, which means that the only part of the algorithm for which the number of iterations depends on the values of the elements of the matrix is the "conquer" portion, solving the secular equation:
1+\sum_j^m w_j^2/(d_j-\lambda)=0
Thus, for each 2x2 block, the problem boils down to how to order the two roots of the secular equation in a way that doesn't actually depend on the values of the original matrix.
The second partial solution is much easier to implement, but more prone to fail. In retrospect, it's also obvious.
Two different eigenvectors of the same matrix will always be orthogonal to each other. Thus, if the eigenvectors smoothly vary as a function of a single parameter a, then:
v_i(a).v_j(a+da) = \delta_{ij} + O(da)
Thus, this gives a natural mapping between eigenvectors as the parameter a varies.
This is similar to the idea David Zwicker and jorgeca suggested of measuring global distance between pairs of eigenvectors, but much easier to implement. However, implementations of this will be prone to failure in regions where the eigenvectors are rapidly varying or if the change in the parameter a is too large.
Also, the question of what happens at crossings of eigenvalues is interesting, because at each such crossing the system becomes degenerate. However, within the set of allowed eigenvectors spanning the degeneracy, there will be two which satisfy the dot product condition and which can be used as the basis spanning the degeneracy, thus maintaining the mapping.
This is, of course, assuming that it is correct to treat the eigenvectors as smooth continuous functions on the parameter space, which (as jorgeca pointed out) I'm not certain can be assumed.
回答2:
If I understand well you are looking for the eigenvalues \lambda(t) of matrices A(t) given by entries that are continuous functions of a real variable t.
Let me to recommend you to consult the books:
[1] R. Bathia: Matrix analysis, Springer. Chapter VI?
[2] T. Kato: A short introduction to perturbation theory of linear operators, Theorem 5.2 of Chapter II.
[3] D. Hinrichsen, A. Pritchard: Mathematical systems theory, Vol. 1, Corollary 4.1.19.
来源：https://stackoverflow.com/questions/17105436/tracking-eigenvectors-of-a-1-parameter-family-of-matrices