Tensorflow predicts always the same result

Question

I\'m trying to get the TensorFlow example running with my own data, but somehow the classifier always picks the same class for every test example. The input data is always s

Answer 1

There are three potential issues in your code:

1. The weights, W, are initialized to zero. This question from stats.stackexchange.com has a good discussion of why this can lead to poor training outcomes (such as getting stuck in a local minimum). Instead, you should initialize them randomly, for example as follows:

   W = tf.Variable(tf.truncated_normal([pixels, labels],
                                       stddev=1./math.sqrt(pixels)))
   

2. The cross_entropy should be aggregated to a single, scalar value before minimizing it, using for example tf.reduce_mean():

   cross_entropy = tf.reduce_mean(
       tf.nn.softmax_cross_entropy_with_logits(y_prime, y_))
   

3. You may get faster convergence if you train on mini-batches (or even single examples) rather than training on the entire dataset at once:

   for i in range(10):
           for j in range(4000):
               res = train_step.run({x: train_images[j:j+1],
                                     y_: train_labels[j:j+1]})