sklearn's PLSRegression: “ValueError: array must not contain infs or NaNs”

Question

When using sklearn.cross_decomposition.PLSRegression:

import numpy as np
import sklearn.cross_decomposition

pls2 = sklearn.cross_decomposition.PLSRegression()
xx = np.random.random((5,5))
yy = np.zeros((5,5) ) 

yy[0,:] = [0,1,0,0,0]
yy[1,:] = [0,0,0,1,0]
yy[2,:] = [0,0,0,0,1]
#yy[3,:] = [1,0,0,0,0] # Uncommenting this line solves the issue

pls2.fit(xx, yy)

I get:

C:\Anaconda\lib\site-packages\sklearn\cross_decomposition\pls_.py:44: RuntimeWarning: invalid value encountered in divide
  x_weights = np.dot(X.T, y_score) / np.dot(y_score.T, y_score)
C:\Anaconda\lib\site-packages\sklearn\cross_decomposition\pls_.py:64: RuntimeWarning: invalid value encountered in less
  if np.dot(x_weights_diff.T, x_weights_diff) < tol or Y.shape[1] == 1:
C:\Anaconda\lib\site-packages\sklearn\cross_decomposition\pls_.py:67: UserWarning: Maximum number of iterations reached
  warnings.warn('Maximum number of iterations reached')
C:\Anaconda\lib\site-packages\sklearn\cross_decomposition\pls_.py:297: RuntimeWarning: invalid value encountered in less
  if np.dot(x_scores.T, x_scores) < np.finfo(np.double).eps:
C:\Anaconda\lib\site-packages\sklearn\cross_decomposition\pls_.py:275: RuntimeWarning: invalid value encountered in less
  if np.all(np.dot(Yk.T, Yk) < np.finfo(np.double).eps):
Traceback (most recent call last):
  File "C:\svn\hw4\code\test_plsr2.py", line 8, in <module>
    pls2.fit(xx, yy)
  File "C:\Anaconda\lib\site-packages\sklearn\cross_decomposition\pls_.py", line 335, in fit
    linalg.pinv(np.dot(self.x_loadings_.T, self.x_weights_)))
  File "C:\Anaconda\lib\site-packages\scipy\linalg\basic.py", line 889, in pinv
    a = _asarray_validated(a, check_finite=check_finite)
  File "C:\Anaconda\lib\site-packages\scipy\_lib\_util.py", line 135, in _asarray_validated
    a = np.asarray_chkfinite(a)
  File "C:\Anaconda\lib\site-packages\numpy\lib\function_base.py", line 613, in asarray_chkfinite
    "array must not contain infs or NaNs")
ValueError: array must not contain infs or NaNs

What could be the issue?

I am aware of scikit-learn GitHub issue #2089, but since I use scikit-learn 0.16.1 (with Python 2.7.10 x64) this problem should be solved (the code snippets mentioned in the GitHub issue work fine).

Answer 1

Please check if any of your values being passed in are NaN or inf:

np.isnan(xx).any()
np.isnan(yy).any()

np.isinf(xx).any()
np.isinf(yy).any()

If any of those yields true. Remove the nan entries or inf entries. E.g. you can set them to 0 with:

xx = np.nan_to_num(xx)
yy = np.nan_to_num(yy)

It's also possible for numpy to be fed such large positive and negative and zeroed values, that the equations deep down in the library are producing zeros, Nan's or Inf's. One workaround, oddly enough, is to send in smaller numbers (say representative numbers between -1 and 1. One way to do this is by standardization, see: https://stackoverflow.com/a/36390482/445131

If none of that solves the problem, then you may be dealing with a low level bug in the library your using, or some sort of singularity in your data. Create an sscce and post it to stackoverflow or create a new bug report on the library maintaining your software.

Answer 2

The issue is caused by a bug in scikit-learn. I reported it on GitHub: https://github.com/scikit-learn/scikit-learn/issues/2089#issuecomment-152753095

Answer 3

I can reproduce the same bug, I silenced this bug by filtering all 0s away

threshold_for_bug = 0.00000001 # could be any value, ex numpy.min
xx[xx < threshold_for_bug] = threshold_for_bug

This silences the bug (i never check the precision difference)

My system info:

numpy-1.11.2
python-3.5
macOS Sierra

Answer 4

You may want to check your weights for negative values, since this error will also be triggered with negative weights.

Answer 5

I found a tricky little solution that worked for me.

I was doing time series featurization through cesium with this code:

timeInput = np.array(timeData)
valueInput = np.array(data)

#Featurizing Data
featurizedData = featurize.featurize_time_series(times=timeInput,
                                                     values=valueInput,
                                                     errors=None,
                                                     features_to_use=featuresToUse)

which was resulting in this error:

ValueError: array must not contain infs or NaNs

for laughs, I checked the lengths and types of the data:

data:
70
<class 'numpy.int32'>

timeData: 
70
<class 'numpy.float64'>

which made sense, because my times were calculated from delta data in ms.

I decided I'd try to convert data types with this one line of code:

valueInput = valueInput.astype(float)

and it worked, resulting in this code:

timeInput = np.array(timeData)
valueInput = np.array(data)
valueInput = valueInput.astype(float)

#Featurizing Data
try:
    featurizedData = featurize.featurize_time_series(times=timeInput,
                                                     values=valueInput,
                                                     errors=None,
                                                     features_to_use=featuresToUse)

if you're getting an error like this, give matching datatypes a shot