How to define a custom accuracy in Keras to ignore samples with a particular gold label?

问题

I want to write in Keras a custom metric (I am using the tensorflow backend) equivalent to categorical_accuracy, but where the output for samples with a particular gold label (in my case 0, from y_true) have to be ignored. For example, if my outputs were:

Pred 1 - Gold 0

Pred 1 - Gold 1

The accuracy would be 1, since samples with the gold label 0 have to be ignored. That said, the function I wrote (and that is not giving the expected results) is:

def my_accuracy(y_true, y_pred):

    mask = K.any(K.not_equal(K.argmax(y_true, axis=-1), 0), axis=-1, keepdims=True)

    masked_y_true = y_true*K.cast(mask, K.dtype(y_true))
    masked_y_pred = y_pred*K.cast(mask, K.dtype(y_pred))

    return keras.metrics.categorical_accuracy(masked_y_true, masked_y_pred)`

Any help is appreciated, thanks!

回答1:

You could try this approach:

def ignore_accuracy_of_class(class_to_ignore=0):
    def ignore_acc(y_true, y_pred):
        y_true_class = K.argmax(y_true, axis=-1)
        y_pred_class = K.argmax(y_pred, axis=-1)

        ignore_mask = K.cast(K.not_equal(y_pred_class, class_to_ignore), 'int32')
        matches = K.cast(K.equal(y_true_class, y_pred_class), 'int32') * ignore_mask
        accuracy = K.sum(matches) / K.maximum(K.sum(ignore_mask), 1)
        return accuracy

    return ignore_acc

来源：https://stackoverflow.com/questions/47270722/how-to-define-a-custom-accuracy-in-keras-to-ignore-samples-with-a-particular-gol