Difference between Dense(2) and Dense(1) as the final layer of a binary classification CNN?

Question

In a CNN for binary classification of images, should the shape of output be (number of images, 1) or (number of images, 2)? Specifically, here are 2 kinds of last layer in a

Answer 1

This first one is the correct solution:

keras.layers.Dense(2, activation = 'softmax')(previousLayer)

Usually, we use the softmax activation function to do classification tasks, and the output width will be the number of the categories. This means that if you want to classify one object into three categories with the labels A,B, or C, you would need to make the Dense layer generate an output with a shape of (None, 3). Then you can use the cross_entropyloss function to calculate the LOSS, automatically calculate the gradient, and do the back-propagation process.

If you want to only generate one value with the Dense layer, that means you get a tensor with a shape of (None, 1) - so it produces a single numeric value, like a regression task. You are using the value of the output to represent the category. The answer is correct, but does not perform like the general solution of the classification task.

Answer 2

The difference is if the class probabilities are independent of each other (multi-label classification) or not.

When there are 2 classes and you generally have P(c=1) + P(c=0) = 1 then

keras.layers.Dense(2, activation = 'softmax') 

keras.layers.Dense(1, activation = 'sigmoid')

both are correct in terms of class probabilities. The only difference being how you supply the labels during training. But

keras.layers.Dense(2, activation = 'sigmoid')

is incorrect in that context. However, it is correct implementation if you have P(c=1) + P(c=0) != 1. This is the case for multi-label classification where an instance may belong to more than one correct class.