Basic function minimisation and variable tracking in TensorFlow 2.0

问题

I am trying to perform the most basic function minimisation possible in TensorFlow 2.0, exactly as in the question Tensorflow 2.0: minimize a simple function, however I cannot get the solution described there to work. Here is my attempt, mostly copy-pasted but with some bits that seemed to be missing added in.

import tensorflow as tf

x = tf.Variable(2, name='x', trainable=True, dtype=tf.float32)
with tf.GradientTape() as t:
    y = tf.math.square(x)

# Is the tape that computes the gradients!
trainable_variables = [x]

#### Option 2
# To use minimize you have to define your loss computation as a funcction
def compute_loss():
    y = tf.math.square(x)
    return y

opt = tf.optimizers.Adam(learning_rate=0.001)
train = opt.minimize(compute_loss, var_list=trainable_variables)

print("x:", x)
print("y:", y)

Output:

x: <tf.Variable 'x:0' shape=() dtype=float32, numpy=1.999>
y: tf.Tensor(4.0, shape=(), dtype=float32)

So it says the minimum is at x=1.999, but obviously that is wrong. So what happened? I suppose it only performed one loop of the minimiser or something? If so then "minimize" seems like a terrible name for the function. How is this supposed to work?

On a side note, I also need to know the values of intermediate variables that are calculated in the loss function (the example only has y, but imagine that it took several steps to compute y and I want all those numbers). I don't think I am using the gradient tape correctly either, it is not obvious to me that it has anything to do with the computations in the loss function (I just copied this stuff from the other question).

回答1:

You need to call minimize multiple times, because minimize only performs a single step of your optimisation.

Following should work

import tensorflow as tf

x = tf.Variable(2, name='x', trainable=True, dtype=tf.float32)

# Is the tape that computes the gradients!
trainable_variables = [x]

# To use minimize you have to define your loss computation as a funcction
class Model():
    def __init__(self):
        self.y = 0

    def compute_loss(self):
        self.y = tf.math.square(x)
        return self.y

opt = tf.optimizers.Adam(learning_rate=0.01)
model = Model()
for i in range(1000):
    train = opt.minimize(model.compute_loss, var_list=trainable_variables)

print("x:", x)
print("y:", model.y)

来源：https://stackoverflow.com/questions/55921207/basic-function-minimisation-and-variable-tracking-in-tensorflow-2-0

标签

python

tensorflow

tensorflow2.0