问题:

Im trying to implement ZCA whitening and found some articles to do it, but they are a bit confusing.. can someone shine a light for me?

Any tip or help is appreciated!

Here is the articles i read :

http://courses.media.mit.edu/2010fall/mas622j/whiten.pdf http://bbabenko.tumblr.com/post/86756017649/learning-low-level-vision-feautres-in-10-lines-of

I tried several things but most of them i didnt understand and i got locked at some step. Right now i have this as base to start again :

dtype = np.float32 data = np.loadtxt("../inputData/train.csv", dtype=dtype, delimiter=',', skiprows=1) img = ((data[1,1:]).reshape((28,28)).astype('uint8')*255)

回答1:

As pointed out in R.M's comment, Andfoy's ZCA whitening function contains a small, but crucial mistake: the np.diag(S) should be removed. Numpy returns S as a m x 1 vector and not a m x m matrix (as is common to other svd implementations, e.g. Matlab). Hence the ZCAMatrix variable becomes a m x 1 vector and not a m x m matrix as it should be (when the input is m x n). (Also, the covariance matrix in Andfoy's answer is only valid if X is pre-centered, i.e mean 0).

Here is a python function for generating the ZCA whitening matrix:

def zca_whitening_matrix(X):     """     Function to compute ZCA whitening matrix (aka Mahalanobis whitening).     INPUT:  X: [M x N] matrix.         Rows: Variables         Columns: Observations     OUTPUT: ZCAMatrix: [M x M] matrix     """     # Covariance matrix [column-wise variables]: Sigma = (X-mu)' * (X-mu) / N     sigma = np.cov(X, rowvar=True) # [M x M]     # Singular Value Decomposition. X = U * np.diag(S) * V     U,S,V = np.linalg.svd(sigma)         # U: [M x M] eigenvectors of sigma.         # S: [M x 1] eigenvalues of sigma.         # V: [M x M] transpose of U     # Whitening constant: prevents division by zero     epsilon = 1e-5     # ZCA Whitening matrix: U * Lambda * U'     ZCAMatrix = np.dot(U, np.dot(np.diag(1.0/np.sqrt(S + epsilon)), U.T)) # [M x M]     return ZCAMatrix

And an example of the usage:

X = np.array([[0, 2, 2], [1, 1, 0], [2, 0, 1], [1, 3, 5], [10, 10, 10] ]) # Input: X [5 x 3] matrix ZCAMatrix = zca_whitening_matrix(X) # get ZCAMatrix ZCAMatrix # [5 x 5] matrix xZCAMatrix = np.dot(ZCAMatrix, X) # project X onto the ZCAMatrix xZCAMatrix # [5 x 3] matrix

Hope it helps!

NB: You can see the full answer, in Python, to the Stanford UFLDL ZCA Whitening exercise here.

回答2:

Is your data stored in an mxn matrix? Where m is the dimension of the data and n are the total number of cases? If that's not the case, you should resize your data. For instance if your images are of size 28x28 and you have only one image, you should have a 1x784 vector. You could use this function:

import numpy as np  def flatten_matrix(matrix):     vector = matrix.flatten(1)     vector = vector.reshape(1, len(vector))     return vector

Then you apply ZCA Whitening to your training set using:

def zca_whitening(inputs):     sigma = np.dot(inputs, inputs.T)/inputs.shape[1] #Correlation matrix     U,S,V = np.linalg.svd(sigma) #Singular Value Decomposition     epsilon = 0.1                #Whitening constant, it prevents division by zero     ZCAMatrix = np.dot(np.dot(U, np.diag(1.0/np.sqrt(np.diag(S) + epsilon))), U.T)                     #ZCA Whitening matrix     return np.dot(ZCAMatrix, inputs)   #Data whitening

It is important to save the ZCAMatrix matrix, you should multiply your test cases if you want to predict after training the Neural Net.

Finally, I invite you to take the Stanford UFLDL Tutorials at http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial or http://ufldl.stanford.edu/tutorial/ . They have pretty good explanations and also some programming exercises on MATLAB, however, almost all the functions found on MATLAB are on Numpy by the same name. I hope this may give an insight.

转载请标明出处:How to implement ZCA Whitening? Python

文章来源: How to implement ZCA Whitening? Python

标签

sigma

python