Do the operations defined in array ops in Tensorflow have gradient defined?

问题

I want to know whether the tensorflow operations in this link, have a gradient defined. I am asking because I am implementing a custom loss function and when I run it I always have this error :

ValueError: An operation has `None` for gradient. Please make sure that all of your ops have a gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.

This is my custom Loss function:

def calculate_additional_loss(y_true,y_pred):
#additional loss
x_decoded_normalized = original_dim* y_pred
#y_true = K.print_tensor(y_true, message='y_true = ')
#y_pred = K.print_tensor(y_pred, message='y_pred = ')
error = tf.constant(0, dtype= tf.float32)
additional_loss= tf.constant(0, dtype= tf.float32)
final_loss= tf.constant(0, dtype= tf.float32)
for k in range(batch_size):
    #add padding
    reshaped_elem_1 = K.reshape(x_decoded_normalized[k], [DIM,DIM])

    a = K.reshape(reshaped_elem_1[:,DIM-1], [DIM,1])
    b = K.reshape(reshaped_elem_1[:,1], [DIM,1])

    reshaped_elem_1 = tf.concat ([b,reshaped_elem_1], axis= 1)
    reshaped_elem_1 = tf.concat ([reshaped_elem_1,a], axis= 1)

    c= K.reshape(reshaped_elem_1[DIM-1,:], [1,DIM+2])
    d= K.reshape(reshaped_elem_1[1,:], [1,DIM+2])
    reshaped_elem_1 = tf.concat ([d,reshaped_elem_1],axis=0)
    reshaped_elem_1 = tf.concat ([reshaped_elem_1,c],axis=0)

    for (i,j) in range(reshaped_elem_1.shape[0],reshaped_elem_1.shape[1]):
        error = tf.add(error, tf.pow((reshaped_elem_1[i,j]- 
                       reshaped_elem_1[i,j+1]),-2), 
                       tf.pow((reshaped_elem_1[i,j]-reshaped_elem_1[i,j- 
                       1]),-2), tf.pow((reshaped_elem_1[i,j]- 
                       reshaped_elem_1[i-1,j]),-2), 
                       tf.pow((reshaped_elem_1[i,j]-reshaped_elem_1[i+1,j]),-2))
    additional_loss = tf.add(additional_loss, tf.divide(error, original_dim))
final_loss += tf.divide(additional_loss, batch_size)
print('final_loss', final_loss)
return final_loss

and This is where I am calling it:

models = (encoder, decoder)
additional_loss = calculate_additional_loss(inputs,outputs)
vae.add_loss(additional_loss)
vae.compile(optimizer='adam')
vae.summary()

plot_model(vae,to_file='vae_mlp.png',show_shapes=True)
vae.fit(x_train, epochs=epochs, batch_size=batch_size, validation_data=(x_test, None), verbose = 1, callbacks=[CustomMetrics()])

Thank you in advance.

回答1:

Most ops have a defined gradient. There are some ops for which a gradient is not defined and the error message you get gives you some examples.

Having said that, there are couple of mistakes I see in your code :

final_loss is defined as tf.constant, but you are trying to increment it.
You are taking a tuple from range
error is defined as tf.constant, but you are trying to increment it.
Don't use for loop in this way over batch_size. Instead use TensorFlow functions to handle batch dimension directly. This way you are just proliferating your nodes.
The way you have written your code makes me think that you're thinking of TensorFlow as pure python. It is not. You define the graph and then you execute it inside a session. So, in the function use TF functions to just define the computations.

来源：https://stackoverflow.com/questions/52261090/do-the-operations-defined-in-array-ops-in-tensorflow-have-gradient-defined

标签

tensorflow

keras

loss-function

gradient