Using all the raw pixel values in an image directly as features aren't great, especially as the number of features increase, due to the very large search space (169 features represents a large search space, which can be difficult for any classification algorithm to solve). This is perhaps why moving to an 20x20 image actually degrades performance compared to 13x13. Reducing your feature set/search space might improve performance since you simplify the classification problem.
A very simple (and generic) approach to accomplish this is to use pixel statistics as features. This is the mean and standard deviation (SD) of the raw pixel values in a given region of the image. This captures the contrast/ brightness of a given region.
You can choose the regions based on trial and error, e.g., these can be:
- a series of concentric circular regions, increasing in radius, in the centre of the image. The mean and SD of four circular regions of increasing size gives eight features.
- a series of rectangular regions, either increasing in size or fixed size but placed around different regions in the image. The mean and SD of four non-overlapping regions (of size 6x6) in the four corners of the image and one in the centre gives 10 features.
- a combination of circular and square regions.
