Change Hyperparameters on an ongoing simulation of TensorFlow

问题

I wonder if it is possible to change some hyperparameters, let's say learning rate or regularization in real time during a TensorFlow simulation.

Something like: You are monitoring the cost function of your neural net(NN) and then you decide that your NN could be doing better if you reduce the regularization term. But you would like to do this without interrupting everything. Just typing the new value in somewhere and then changing the regularization in the next epoch, for example.

回答1:

You could just declare your hyper parameters has placeholder or a not trainable Variable, and change them as you need it using the feed_dict.

lr = tf.get_variable('learning_rate',initializer=tf.constant(1), trainable=False)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print(sess.run(lr)) # prints 1
    print(sess.run(lr, {lr: 10})) # prints 10

来源：https://stackoverflow.com/questions/49011305/change-hyperparameters-on-an-ongoing-simulation-of-tensorflow