LOSS not changeing in very simple KERAS binary classifier

问题

I'm trying to get a very (over) simplified Keras binary classifier neural network running without success. The LOSS just stays constant. I've played around with Optimizers (SGD, Adam, RMSProp), Learningrates, Weight-Initializations, Batch Size and input data normalization so far.

Nothing changes at all. Am I doing something fundamentally wrong? Here is the code:

from tensorflow import keras
from keras import Sequential
from keras.layers import Dense
from keras.optimizers import SGD

data = np.array(
    [
        [100,35,35,12,0],
        [101,46,35,21,0],
        [130,56,46,3412,1],
        [131,58,48,3542,1]
    ]
)

x = data[:,1:-1]
y_target = data[:,-1]

x = x / np.linalg.norm(x)

model = Sequential()
model.add(Dense(3, input_shape=(3,), activation='softmax', kernel_initializer='lecun_normal',
                bias_initializer='lecun_normal'))
model.add(Dense(1, activation='softmax', kernel_initializer='lecun_normal',
                bias_initializer='lecun_normal'))

model.compile(optimizer=SGD(learning_rate=0.1),
              loss='binary_crossentropy',
              metrics=['accuracy'])

model.fit(x, y_target, batch_size=2, epochs=10,
          verbose=1)

回答1:

Softmax definition is:

exp(a) / sum(exp(a)

so when you use with a single neuron you will get:

exp(a) / exp(a) = 1

That is why your classifier doesn't work with a single neuron.

You can use sigmoid instead in this special case:

exp(a) / (exp(a) + 1)

Furthermore sigmoid function is for two class classifiers. Softmax is an extension of sigmoid for multiclass classifers.

For the first layer you should use relu or sigmoid function instead of softmax.

回答2:

This is the working solution based on the feedback I got

from tensorflow import keras
from keras import Sequential
from keras.layers import Dense
from keras.optimizers import SGD
from keras.utils import to_categorical

data = np.array(
    [
        [100,35,35,12,0],
        [101,46,35,21,0],
        [130,56,46,3412,1],
        [131,58,48,3542,1]
    ]
)

x = data[:,1:-1]
y_target = data[:,-1]

x = x / np.linalg.norm(x)

model = Sequential()
model.add(Dense(3, input_shape=(3,), activation='sigmoid'))
model.add(Dense(1, activation='sigmoid'))

model.compile(optimizer=SGD(learning_rate=0.1),
              loss='binary_crossentropy',
              metrics=['accuracy'])

model.fit(x, y_target, epochs=1000,
          verbose=1)

来源：https://stackoverflow.com/questions/59129802/loss-not-changeing-in-very-simple-keras-binary-classifier

标签

python

tensorflow

keras

classification