Determinism in tensorflow gradient updates?

Question

So I have a very simple NN script written in Tensorflow, and I am having a hard time trying to trace down where some \"randomness\" is coming in from.

I have record

Answer 1

I have faced the same problem.. The working solution for me was to:

1- use tf.set_random_seed(1) in order to make all tf functions have the same seed every new run

2- Training the model using CPU not the GPU to avoid GPU non-deterministic operations due to precision.