What's the difference between Variable and ResourceVariable in Tensorflow

ε祈祈猫儿з 提交于 2019-12-20 18:44:11

问题


In tensorflow, Variable is a resource, inherited from ResourceBase and managed by ResourceMgr. But why there is another thing named ResourceVariable? Both of them can be used for optimizers like gradient_descent(see this example). What's the difference? I know the former one is well documented and most often used. What's the purpose of the later one?


回答1:


ResourceVariable is the replacement for Variable, that aims to clean up some of the messier aspects of the semantics of Variable.

ResourceVariable is the default in TF 2.0 and you very likely don't care about the differences between the two unless you are working on details deep inside the Tensorflow implementation. When eager execution is enabled tf.Variable also creates resource variables.

So just use tfVariable for now, it's almost certainly what you want; if you experience issues which look like race conditions or bugs from inconsistent values of variables in code you can try enabling resource variables (by either passing use_resource=True to your variable-creating code or calling tf.enable_resource_variables() in TF 1.x).




回答2:


From the comments:

Unlike tf.Variable, a tf.ResourceVariable has well-defined semantics. Each usage of a ResourceVariable in a TensorFlow graph adds a read_value operation to the graph. The Tensors returned by a read_value operation are guaranteed to see all modifications to the value of the variable which happen in any operation on which the read_value depends on (either directly, indirectly, or via a control dependency) and guaranteed to not see any modification to the value of the variable on which the read_value operation does not depend on. For example, if there is more than one assignment to a ResourceVariable in a single session.run call there is a well-defined value for each operation which uses the variable's value if the assignments and the read are connected by edges in the graph. Consider the following example, in which two writes can cause tf.Variable and tf.ResourceVariable to behave differently:

a = tf.ResourceVariable(1.0)
a.initializer.run()
assign = a.assign(2.0)
with tf.control_dependencies([assign]):
    b = a.read_value()
with tf.control_dependencies([b]):
    other_assign = a.assign(3.0)
with tf.control_dependencies([other_assign]):
    # Will print 2.0 because the value was read before other_assign ran. If
    # `a` was a tf.Variable instead, 2.0 or 3.0 could be printed.
    tf.Print(b, [b]).eval()

To enforce these consistency properties tf.ResourceVariable might make more copies than an equivalent tf.Variable under the hood, so tf.Variable is still not deprecated.



来源:https://stackoverflow.com/questions/40817665/whats-the-difference-between-variable-and-resourcevariable-in-tensorflow

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!