Computing CCA through three approaches

问题

I have recently studied the concepts of CCA and wanted to implement it in MATLAB. However there is an existing matlab command canoncorr present. I wanted to write my own code. I have studied it extensively and found three approaches :

1: Hardoon : The approach uses lagrange multipliers to decompose the problem into an generalised eigenvalue problem. The code can be found here : cca_hardoon For sanity sake I am also giving the code here : The data has to be centered previously.

function [Wx, Wy, r] = cca(X,Y)

% CCA calculate canonical correlations
%
% [Wx Wy r] = cca(X,Y) where Wx and Wy contains the canonical correlation
% vectors as columns and r is a vector with corresponding canonical
% correlations.
%
% Update 31/01/05 added bug handling.

if (nargin ~= 2)
  disp('Inocorrect number of inputs');
  help cca;
  Wx = 0; Wy = 0; r = 0;
  return;
end


% calculating the covariance matrices
z = [X; Y];
C = cov(z.');
sx = size(X,1);
sy = size(Y,1);
Cxx = C(1:sx, 1:sx) + 10^(-8)*eye(sx);
Cxy = C(1:sx, sx+1:sx+sy);
Cyx = Cxy';
Cyy = C(sx+1:sx+sy,sx+1:sx+sy) + 10^(-8)*eye(sy);

%calculating the Wx cca matrix
Rx = chol(Cxx);
invRx = inv(Rx);
Z = invRx'*Cxy*(Cyy\Cyx)*invRx;
Z = 0.5*(Z' + Z);  % making sure that Z is a symmetric matrix
[Wx,r] = eig(Z);   % basis in h (X)
r = sqrt(real(r)); % as the original r we get is lamda^2
Wx = invRx * Wx;   % actual Wx values

% calculating Wy
Wy = (Cyy\Cyx) * Wx; 

% by dividing it by lamda
Wy = Wy./repmat(diag(r)',sy,1);

2. MATLAB approach Please note the centering of data is done within the code itself.

3. CCA by Normal SVD only : This approach does not require the qr decomposition and utilizes the svd decomposition only. I have referred top this article here : cca by svd. Please refer to the text articles below which are taken from the referred article.

I have tried to code this program myself but unsuccessfully.

function [A,B,r,U,V] = cca_by_svd(x,y)
% computing the means
N = size(x,1); mu_x = mean(x,1); mu_y = mean(y,1);
% substracting the means
x = x - repmat(mu_x,N,1); y = y - repmat(mu_y,N,1);

x = x.'; y = y.';
% computing the covariance matrices
Cxx = (1/N)*x*(x.');  Cyy = (1/N)*y*(y.'); Cxy = (1/N)*x*(y.');

%dimension
m = min(rank(x),rank(y));
%m = min(size(x,1),size(y,1));

% computing the quare root inverse of the matrix
[V,D]=eig(Cxx); d = diag(D);
% Making all the eigen values positive
d = (d+abs(d))/2; d2 = 1./sqrt(d); d2(d==0)=0; Cxx_iv=V*diag(d2)*inv(V);

% computing the quare root inverse of the matrix
[V,D]=eig(Cyy); d = diag(D);
% Making all the eigen values positive
d = (d+abs(d))/2; d2 = 1./sqrt(d); d2(d==0)=0; Cyy_iv=V*diag(d2)*inv(V);

Omega = Cxx_iv*Cxy*Cyy_iv;
[C,Sigma,D] = svd(Omega);
A = Cxx_iv*C; A = A(:,1:m);
B = Cyy_iv*D.'; B = B(:,1:m);
A = real(A); B = real(B);
U = A.'*x; V = B.'*y;
r = Sigma(1:m,1:m);

I am running this code snippet:

clc;clear all;close all;
load carbig;
X = [Displacement Horsepower Weight Acceleration MPG];
nans = sum(isnan(X),2) > 0;
x = X(~nans,1:3);
y = X(~nans,4:5);
[A1, B1, r1, U1, V1] = canoncorr(x, y);

[A2, B2, r2, U2, V2] = cca_by_svd(x, y);

[A3, B3, r3] = cca(x.',y.',1);

The projection vector is coming out to be this :

>> A1
A1 =
    0.0025    0.0048
    0.0202    0.0409
   -0.0000   -0.0027
>> A2
A2 =
    0.0025    0.0048
    0.0202    0.0410
   -0.0000   -0.0027
>> A3
A3 =
   -0.0302   -0.0050   -0.0022
    0.0385   -0.0420   -0.0176
    0.0020    0.0027   -0.0001
>> B1
B1 =
   -0.1666   -0.3637
   -0.0916    0.1078
>> B2
B2 =
   -0.1668   -0.3642
   -0.0917    0.1079
>> B3
B3 =
   0.0000 + 0.0000i   0.3460 + 0.0000i   0.1336 + 0.0000i
   0.0000 + 0.0000i  -0.0967 + 0.0000i   0.0989 + 0.0000i

Question: Can someone please tell me where I am going wrong. The three approaches that I have referred all solve the same problem and ideally their solutions should converge. I admit my code 'cca_by_svd' may be wrong but hardoon's code and matlab's output should be same. Please point out to me where I am going wrong. edit I have rechecked and corrected my code. Now for this dataset the method 2 and 3 converge.

回答1:

There's a few things that cca(X,Y) doesn't do that canoncorr does:

One is normalizing the data. If you add X = normc(X')' (also for Y) to your cca(X,Y) function, the output r will match that of canoncorr. If you look into canoncorr's code, you'll see that it starts by QR decomposition of X and Y.

Another difference is that eig sorts the eigenvalues in ascending order, so cca(X,Y) should flip the output of eig(Z).

NOTE: Despite correcting these differences, I wasn't able to fully recover Wx and Wy to match the outputs of canoncorr. Ideally, Wx'*Wx should look exactly alike between cca and canoncorr.

来源：https://stackoverflow.com/questions/30168362/computing-cca-through-three-approaches

标签

matlab

correlation