I'm trying to run a LSTM, and when I use the code below:
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy', 'f1score', 'precision', 'recall'])
ValueError: ('Unknown metric function', ':f1score').
I've done my searches and found this url: https://github.com/fchollet/keras/issues/5400
The "metrics" in the "model.compile" part in this url is exactly the same as mine, and no errors are returned.
I suspect you are using Keras 2.X. As explained in https://keras.io/metrics/, you can create custom metrics. These metrics appear to take only
(y_true, y_pred) as function arguments, so a generalized implementation of fbeta is not possible.
Here is an implementation of
f1_score based on the keras 1.2.2 source code.
import keras.backend as K def f1_score(y_true, y_pred): # Count positive samples. c1 = K.sum(K.round(K.clip(y_true * y_pred, 0, 1))) c2 = K.sum(K.round(K.clip(y_pred, 0, 1))) c3 = K.sum(K.round(K.clip(y_true, 0, 1))) # If there are no true samples, fix the F1 score at 0. if c3 == 0: return 0 # How many selected items are relevant? precision = c1 / c2 # How many relevant items are selected? recall = c1 / c3 # Calculate f1_score f1_score = 2 * (precision * recall) / (precision + recall) return f1_score
To use, simply add
f1_score to your list of metrics when you compile your model, after defining the custom metric. For example:
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy',f1_score])
K.epsilon() works well in this code. You can use this in the definition of c1, c2, and c3.