TensorFlow: Is there a metric to calculate and update top k accuracy?

Question

The current tf.contrib.metrics.streaming_accuracy is only able to calculate the top 1 accuracy, and not the top k. As a workaround, this is what I've been using:

tf.reduce_mean(tf.cast(tf.nn.in_top_k(predictions=predictions, targets=labels, k=5), tf.float32))

However, this does not give me a way to calculate the streaming accuracies averaged across each batch, which would be useful in getting a stable evaluation accuracy. I am currently manually calculating this streaming top 5 accuracy through using its numpy output, but this means I won't be able to visualize this metric on tensorboard.

Is there a way to have a simpler implementation by creating an accuracy_update function, or is there an existing function that already does this?

Thank you.

Answer 1

You could replace your use of tf.contrib.metrics.streaming_accuracy by the lower-level tf.metrics.mean, which is by the way ultimately used by streaming_accuracy -- you will find a similarity in their respective documentations.

E.g. (not tested)

tf.metrics.mean(tf.nn.in_top_k(predictions=predictions, targets=labels, k=5))

Answer 2

For top-k accuracy per batch, this also works.

k_val=3
accs = []
for each_bach in range(batch_size):
    acc = tf.keras.metrics.top_k_categorical_accuracy(y_true=tf_class1[each_bach], y_pred=tf_class2[each_bach], k=k_val)
    accs.append(acc)

acc_data_per_batch = tf.reduce_mean(accs)

tf.keras.metrics.top_k_categorical_accuracy returns K.mean( nn.in_top_k(y_pred, math_ops.argmax(y_true, axis=-1), k), axis=-1) per batch