Easy ways to detect and crop blocks (paragraphs) of text out of image?

Question

I have done some research on the subject, but I think my question is significantly different from what has been asked before.

My PhD thesis deals with OCR-ing an old

Answer 1

I have a few ideas to share... I think I would proceed along these lines:

LOW-RESOLUTION COPY OF ORIGINAL IMAGE JUST FOR REFERENCE

Step 1 - Threshold to Black and White

I think I would use OpenCV's Otsu thresholding for this.

Step 2 - Find vertical black line

I would average the pixels in every column of the image and find the one with the lowest average and that should be the vertical line up the middle. Code below outputs:

Centreline at column: 1635

Step 3 - Split image in two and trim excess white space

Step 4 - Box filter

I would box filter with a 55x45 box that matches the indent at the start of each paragraph then threshold so all paragraph starts are marked with black boxes.

I am pretty new to OpenCV but have coded the above ideas as follows - I m sure lots of it could be made more robust and more efficient so treat it as conceptual ;-)

#include 
#include 

using namespace cv;
using namespace std;

int
main(int argc,char*argv[])
{
   // Load image
   Mat orig=imread("page.png",IMREAD_COLOR);

   vector PNGwriteOptions;
   PNGwriteOptions.push_back(CV_IMWRITE_PNG_COMPRESSION);
   PNGwriteOptions.push_back(9);

   // Get greyscale and Otsu-thresholded version
   Mat bw,grey;
   cvtColor(orig,grey,CV_RGB2GRAY);
   threshold(grey,bw,0,255,CV_THRESH_BINARY|CV_THRESH_OTSU);

   // Find vertical centreline by looking for lowest column average - i.e. darkest vertical bar
   Mat colsums;
   reduce(bw,colsums,0,CV_REDUCE_AVG);
   double min,max;
   Point min_loc, max_loc;
   minMaxLoc(colsums,&min,&max,&min_loc,&max_loc);
   cout << "Centreline at column: " << min_loc.x << endl;

   namedWindow("test",CV_WINDOW_AUTOSIZE);

   // Split image into left and right
   Rect leftROI(0,0,min_loc.x,bw.rows);
   Mat  leftbw=bw(leftROI);
   Rect rightROI(min_loc.x+8,0,bw.cols-min_loc.x-8,bw.rows);
   Mat  rightbw=bw(rightROI);
   imshow("test",leftbw);
   waitKey(0); 
   imshow("test",rightbw);
   waitKey(0); 

   // Trim surrounding whitespace off
   Mat Points;
   Mat inverted =  cv::Scalar::all(255) - leftbw;
   findNonZero(inverted,Points);
   Rect bRect=boundingRect(Points);
   Mat lefttrimmed=leftbw(bRect);

   inverted =  cv::Scalar::all(255) - rightbw;
   findNonZero(inverted,Points);
   bRect=boundingRect(Points);
   Mat righttrimmed=rightbw(bRect);

   imwrite("lefttrimmed.png",lefttrimmed,PNGwriteOptions);
   imwrite("righttrimmed.png",righttrimmed,PNGwriteOptions);

   // Box filter with 55x45 rectangle to match size of paragraph indent on left
   Mat lBoxFilt,rBoxFilt;
   boxFilter(lefttrimmed,lBoxFilt,-1,Size(55,45));
   normalize(lBoxFilt,lBoxFilt,0,255,NORM_MINMAX,CV_8UC1);
   threshold(lBoxFilt,lBoxFilt,254,255,THRESH_BINARY_INV);
   imwrite("leftBoxed.png",lBoxFilt,PNGwriteOptions);

}

Just in case you need a hand to build this code - as it seems non-trivial to compile and link anything against it - I made my CMakeLists.txt file like this and stored it in the same directory as the source file. Then I create a sub-directory called build to do an "out-of-source" build in and the build process is:

cd build
cmake ..
make -j 8
./demo

CMakeLists.txt

cmake_minimum_required(VERSION 2.8)
project(demo)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
find_package(OpenCV)
add_executable(demo main.cpp)
target_link_libraries(demo ${OpenCV_LIBS})

Keywords: Image processing, book, margin, spine, centreline, page, crease, fold, gutter, binding, stitching, text, paragraph, detect, detection.