Keras custom loss function: variable with shape of batch_size (y_true)

问题

When implementing a custom loss function in Keras, I require a tf.Variable with the shape of the batch size of my input data (y_true, y_pred).

def custom_loss(y_true, y_pred):

    counter = tf.Variable(tf.zeros(K.shape(y_true)[0], dtype=tf.float32))
    ...

However, this produces the error:

You must feed a value for placeholder tensor 'dense_17_target' with dtype float and shape [?,?]

If I fix the batch_size to a value:

def custom_loss(y_true, y_pred):

    counter = tf.Variable(tf.zeros(batch_size, dtype=tf.float32))
    ...

such that |training_set| % batch_size and |val_set| % batch_size are equal to zero, everything works fine.

Are there any suggestions, why the assignment of the variable with batch size based on the shape of the input (y_true and y_pred) does not work?

SOLUTION

I found a satisfying solution that works. I initialized the variable with the max batch_size possible (specified during model build time) and used the K.shape(y_true)[0] only for slicing the variable. That way it works perfectly. Here the code:

def custom_loss(y_true, y_pred):
    counter = tf.Variable(tf.zeros(batch_size, dtype=tf.float32))
    ...
    true_counter = counter[:K.shape(y_true)[0]]
    ...

回答1:

It does not work because K.shape returns you a symbolic shape, which is a tensor itself, not a tuple of int values. To get the value from a tensor, you have to evaluate it under a session. See documentation for this. To get a real value prior to evaluation time, use K.int_shape: https://keras.io/backend/#int_shape

However, K.int_shape also not gonna work here, as it is just some static metadata and won't normally reflect the current batch size, but has a placeholder value None.

The solution you found (have a control over the batch size and use it inside the loss) is indeed a good one.

I believe the problem is because you need to know the batch size at the definition time to build the Variable, but it will be known only during the session run time.

If you were working with it as with a tensor, it should be ok, see this example.

回答2:

An alternative solution is to create a Variable and change its shape dynamically using tf.assign with validate_shape=False:

counter = tf.Variable(0.0)
...
val = tf.zeros(tf.shape(y_true)[:1], 0.0)
counter = tf.assign(counter, val, validate_shape=False)

来源：https://stackoverflow.com/questions/54388842/keras-custom-loss-function-variable-with-shape-of-batch-size-y-true

标签

python

tensorflow

keras

deep-learning

loss-function