math domain error while using PCA

Question

I am using python's scikit-learn package to implement PCA .I am getting math

domain error :
C:\Users\Akshenndra\Anaconda2\lib\site-packages\sklearn\decomposition\pca.pyc in _assess_dimension_(spectrum, rank, n_samples, n_features)
     78         for j in range(i + 1, len(spectrum)):
     79             pa += log((spectrum[i] - spectrum[j]) *
---> 80                       (1. / spectrum_[j] - 1. / spectrum_[i])) + log(n_samples)
     81 
     82     ll = pu + pl + pv + pp - pa / 2. - rank * log(n_samples) / 2.

ValueError: math domain error

I already know that math domain error is caused when we take logarithm of a negative number ,but I don't understand here how can there be a negative number inside the logarithm ? because this code works fine for other datasets. maybe is this related to what is written in the sci-kitlearn's website -"This implementation uses the scipy.linalg implementation of the singular value decomposition. It only works for dense arrays and is not scalable to large dimensional data."(there are large number of 0 values)

Answer 1

I think you should add 1 instead, as the numpy log1p description page. Since log(p+1) = 0 when p = 0 (while log(e-99) = -99), and as the quote in the link

For real-valued input, log1p is accurate also for x so small that 1 + x == 1 in floating-point accuracy

The code can be modified as follows to make what you trying to resolve more reasonable:

for i in range(rank):
    for j in range(i + 1, len(spectrum)):
        pa += log((spectrum[i] - spectrum[j]) *
        (1. / spectrum_[j] - 1. / spectrum_[i]) + 1) + log(n_samples + 1)
    ll = pu + pl + pv + pp - pa / 2. - rank * log(n_samples + 1) / 2

Answer 2

I don't know whether i am right or not, but I truly find a way to solve it.

I just print some error information(The value of spectrum_[i] and spectrum_[j]), and I find :

sometimes, they are same!!!

(Maybe they are not same but they are too close, I guess)

so , here

pa += log((spectrum[i] - spectrum[j]) *
                  (1. / spectrum_[j] - 1. / spectrum_[i])) + log(n_samples)

it will report error when calculate log(0).

My way to solve it is to add a very small number 1e-99 to 0, so it become log(0 + 1e-99)

so you can just change it to:

            pa += log((spectrum[i] - spectrum[j]) *
                  (1. / spectrum_[j] - 1. / spectrum_[i]) + 1e-99) + log(n_samples)