I have grid N × M in which each cell is coloured with one colour.
When the player clicks on any cell of the grid of colour α, the cell in the top-leftmost corner of
Note that the cells are either their original colour, or the last selected colour.
This means that we can represent the current state of the board by means of 20 bits (marking for each of the 4*5 cells whether they contain the original colour), and a number in the range 0 to 9 giving the last colour chosen.
This results in a maximum of 10 million states to explore. The backtracking function can avoid having to recurse if it reaches a state that has already been visited. I would expect this change to make your solution considerably faster.
Representing the state by a 20bit mask and the last colour also makes the state a lot quicker to update and restore as only 2 numbers need to be changed instead of memcpy of the whole board.