Recognize a number from an image

Question

I\'m trying to write an application to find the numbers inside an image and add them up.

How can I identify the written number in an image?

Answer 1

In most image processing problems you want to look to leverage as much information you have as possible. Given the image there are assumptions we can make (and possibly more):

1. The boxes around the numbers are consistent.
2. The number on the right is always 8 (or known ahead of time)
3. The number on the left is always a number
4. The number on the left is always handwriting and written by the same person

Then we can simplify the problem using those assumptions:

1. You can use a simpler approach to find the numbers (template matching). When you have the coordinates of the match you can create a sub image and subtract out the template and be left with only the numbers you want to give to the OCR engine. http://docs.opencv.org/doc/tutorials/imgproc/histograms/template_matching/template_matching.html .
2. If you know what numbers to expect, then you can get those from another source and not risk OCR errors. You could even include the 8 as part of the template.
3. You can greatly reduce the vocabulary (possible OCR results), based on this, increasing the OCR engine's accuracy. There is a whitelist setting for TesseractOCR to do this (see https://code.google.com/p/tesseract-ocr/wiki/FAQ#How_do_I_recognize_only_digits?).
4. Handwriting is much harder for an OCR engine to recognize (They are meant for printed fonts). However, you can train the OCR engine to recognize the author's "font". (see http://michaeljaylissner.com/posts/2012/02/11/adding-new-fonts-to-tesseract-3-ocr-engine/)

The gist though is to use any assumptions that you can to reduce the problem into smaller, simpler sub problems. Then look to see what tools are available to solve each of those sub problems individually.

Assumptions are hard to make as well if you have to start worrying about the real world, like if these will be scanned it, you'll need to consider skew or rotation of the "template" or the numbers.