Working with multiple graphs in TensorFlow

Question

Can someone explain to me how name_scope works in TensorFlow?

Suppose I have the following code:

Answer 1

I have written two snippets to do what you are trying to do.

The first switches between graphs and does NOT use name_scope. The second uses the default_graph for both ops and uses name_scope to switch between them...

import tensorflow as tf

g1 = tf.Graph()
with g1.as_default() as g:
  matrix1 = tf.constant([[3., 3.]])
  matrix2 = tf.constant([[2.],[2.]])
  tf.matmul(matrix1, matrix2, name="productX")

g2 = tf.Graph()
with g2.as_default() as g:
  matrix1 = tf.constant([[4., 4.]])
  matrix2 = tf.constant([[5.],[5.]])
  tf.matmul(matrix1, matrix2, name="productX")

product_op = g1.get_tensor_by_name(  "productX:0" )
with tf.Session( graph = g1 ) as sess:
    result = sess.run( product_op )
    print( result )

product_op = g2.get_tensor_by_name(  "productX:0" )
with tf.Session( graph = g2 ) as sess:
  result = sess.run( product_op )
  print( result )

NOTE A) I have taken out the resetting of the default graph (the default graph is never used) and B) I have gotten rid of the 'product' variable and given the operation a name, instead

The important switching code is...

product_op = g2.get_tensor_by_name(  "productX:0" )

where we are using the variable name of the two graphs to distinguish between the two product ops.

NOTE the annoying ':0' that you have to put at the end of the variable name.

And now using name_scope...

import tensorflow as tf

g = tf.get_default_graph()

matrix1 = tf.constant([[3., 3.]])
matrix2 = tf.constant([[2.],[2.]])
with g.name_scope("prodA"):
  tf.matmul(matrix1, matrix2, name="productX")

matrix1 = tf.constant([[4., 4.]])
matrix2 = tf.constant([[5.],[5.]])
with g.name_scope("prodB"):
  tf.matmul(matrix1, matrix2, name="productX")

with tf.Session() as sess:

  product_op = g.get_tensor_by_name(  "prodA/productX:0" )
  result = sess.run( product_op )
  print( result )

  product_op = g.get_tensor_by_name(  "prodB/productX:0" )
  result = sess.run( product_op )
  print( result )

Now the switching line of code is this...

product_op = g.get_tensor_by_name(  "prodB/productX:0" )

NOTE A) There is no switching between graphs or sessions. B) name_scope give you a kind of directory structure looking name hierarchy.

What are the pros and cons of graph vs name_scope switching? If someone wrote a blog on that, I would read it!

I don't imagine that switching between graphs and sessions can be very fast, however.