How to write a custom loss function in Tensorflow?

Question

I am new to tensorflow. I want to write my own custom loss function. Is there any tutorial about this? For example, the hinge loss or a sum_of_square_loss(thoug

Answer 1

Almost in all tensorflow tutorials they use custom functions. For example in the very beginning tutorial they write a custom function:

sums the squares of the deltas between the current model and the provided data

squared_deltas = tf.square(linear_model - y)
loss = tf.reduce_sum(squared_deltas)

In the next MNIST for beginners they use a cross-entropy:

cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))

---

As you see it is not that hard at all: you just need to encode your function in a tensor-format and use their basic functions.

For example here is how you can implement F-beta score (a general approach to F1 score). Its formula is:

The only thing we will need to do is to find how to calculate true_positive, false_positive, false_negative for boolean or 0/1 values. If you have vectors of 0/1 values, you can calculate each of the values as:

TP = tf.count_nonzero(actual, predicted)
FP = tf.count_nonzero((actual - 1) * predicted)
FN = tf.count_nonzero((predicted - 1) * actual)

Now once you know these values you can easily get your

denom = (1 + b**2) * TP + b**2 TN + FP
Fb = (1 + b**2) * TP / denom