问题
When we finetune a model on a different task, only a part of vars in the model are restored from the pretrained task and others are left as initial values.
As many docs recommends(page1 page2), when training with a local graph, restoring the pretrained model after running the global init op(call restoring in "init_fn" if MonitoredSession or supervisor is included).
But in the distributed case, does global init op make "model_ready" returns true before the restoring-model called? other non-chief nodes will use the "not ready" values.
回答1:
Figure it out. global_variables_initializer is in facet variable_initializers(global_variables()). So we can initialize only some selected vars and restore the lefts from pretrained models. "model_ready" will keep as False until all vars are restored.
来源：https://stackoverflow.com/questions/43883062/whats-the-recommend-way-of-restoring-only-parts-model-in-distributed-tensorflow