There's a similar question here: Tensorflow: how to save/restore a model?
TLDR; you need to recreate model structure using same sequence of TensorFlow API commands before using Saver object to restore the weights
This is suboptimal, follow Github issue #696 for progress on making this easier