How do I get the gradient of the loss at a TensorFlow variable?

Question

The feature I\'m after is to be able to tell what the gradient of a given variable is with respect to my error function given some data.

One way to do this would be

Answer 1

In TensorFlow 2.0 you can use GradientTape to achieve this. GradientTape records the gradients of any computation that happens in the context of that. Below is an example of how you might do that.

import tensorflow as tf

# Here goes the neural network weights as tf.Variable
x = tf.Variable(3.0)

# TensorFlow operations executed within the context of
# a GradientTape are  recorded for differentiation 
with tf.GradientTape() as tape:
  # Doing the computation in the context of the gradient tape
  # For example computing loss
  y = x ** 2 

# Getting the gradient of network weights w.r.t. loss
dy_dx = tape.gradient(y, x) 
print(dy_dx)  # Returns 6