Keras fitting ignoring nan values

白昼怎懂夜的黑 提交于 2019-12-12 19:16:10

问题


I am training a neural network to do regression, (1 input and 1 output). Let's x and y be the usual input and output dataset, respectively.

My problem is that the y dataset (not the x) have some values set to nan, so the fitting goes to nan. I wonder if there is an option to ignore the nan values in the fitting, in a similar way to the numpy functions np.nanmean to calculate the mean ignoring nans and so on.

If that option does not exist I suppose I would have to find the nan values and erase them manually, and at the same time erase the values in x corresponding to the nan position in y.

x       y
2       4
3       2
4      np.nan
5       7
6      np.nan
7      np.nan

In this simple example the nan values in the y column should be removed and at the same time the corresponding values in the x column (4, 6, 7).

Thank you.

EDIT: Ok, I have a problem filtering the nans, I do:

for index, x in np.ndenumerate(a):
    if x == np.nan:
        print index, x

and it doesn't print anything and I am sure there are nan values...

EDIT (SELF ANSWER): Ok, I have found a way to localize the nans:

for index, x in np.ndenumerate(a):
      if x != x:
           print index, x

回答1:


As said in the comments, simply remove the nan as a preprocessing step:

import numpy as np

x = range(2,8)
y = [4,2,np.nan,7,np.nan,np.nan]

for a,b in zip(x,y):
    if str(b) == 'nan':
        x.remove(a)
        y.remove(b)

print x,y

produces [2, 3, 5] [4, 2, 7].

If you're using some tool to preprocess the data which gives you the np.nan, check whether the API allows you to disable this behavior and take a minute to think whether this is really the behavior you want (or if you e.g. want to map this to constants because you find your input to be valuable even though they have no labels).



来源:https://stackoverflow.com/questions/37747886/keras-fitting-ignoring-nan-values

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!