creating a bounding box around a field of optical flow paths

问题

I have used cv::calcOpticalFlowFarneback to calculate the optical flow in the current and previous frames of video with ofxOpenCv in openFrameworks.

I then draw the video with the optical flow field on top and then draw vectors showing the flow of motion in areas that are above a certain threshold.

What I want to do now is create a bounding box of those areas of motion and get the centroid and store that x,y position in a variable for tracking.

This is how I'm drawing my flow field if that helps.

if (calculatedFlow){
    ofSetColor( 255, 255, 255 );
    video.draw( 0, 0);
    int w = gray1.width;
    int h = gray1.height;
    //1. Input images + optical flow
    ofPushMatrix();
    ofScale( 4, 4 );
    //Optical flow
    float *flowXPixels = flowX.getPixelsAsFloats();
    float *flowYPixels = flowY.getPixelsAsFloats();
    ofSetColor( 0, 0, 255 );
    for (int y=0; y<h; y+=5) {
        for (int x=0; x<w; x+=5) {
            float fx = flowXPixels[ x + w * y ];
            float fy = flowYPixels[ x + w * y ];
            //Draw only long vectors
            if ( fabs( fx ) + fabs( fy ) > .5 ) {
                ofDrawRectangle( x-0.5, y-0.5, 1, 1 );
                ofDrawLine( x, y, x + fx, y + fy );
                }
              }
            }
          }

回答1:

For what you are asking, there is no simple answer. Here is a suggested solution. It involves multiple steps, but if your domain is simple enough, you could simplify this.

For each frame,

Calculate flow as two images flow_x,flow_y comparing current frame with previous frame using farneback method.(you seem to be doing this, in your code)
Translate the flow images into an hsv image, where the hue component of each pixel denotes the angle of the flow atan2(flow_y/flow_x) and value component of each pixel denotes the magnitude of the flow sqrt(flow_x\*\*2 + flow_y\*\*2)
In the above step, use your thresholding mechanism to suppress flow- pixels (make them black) whose magnitude falls below a certain threshold.
Segment the HSV image based on color ranges. You could use apriori information about your domain, or you could take histogram of hue components and identify prominent ranges of hues to classify pixels. As a result of this step, you can assign a class to each pixel.
Separate the pixels belonging to each class into multiple images. All pixels belonging to segmented class-1 will goto image-1, all pixels belonging to segmented class-2 will go to image-2 etc. Now each segmented image contains pixels in the HSV image, in a particular color range.
Transform each segmented image as a black and white image, and using opencv's morphological operations split into multiple regions using connectivity. (connected components).
Find the centroid of each connected component.

I found this reference to be helpful in this context.

回答2:

I resolved my problem by creating new image from my flowX, and flowY. This was done by adding flowX and flowY to a new CV FloatImage.

flowX +=flowY;
flowXY = flowX;

Then I was able to do the contour finding from the pixels of the newly created image and then I could store all the centroids of all the blobs of movement.

Like so:

contourFinder.findContours( mask, 10, 10000, 20, false );
//Storing the objects centers with contour finder.
vector<ofxCvBlob>  &blobs = contourFinder.blobs;
int n = blobs.size();     //Get number of blobs
obj.resize( n );          //Resize obj array
for (int i=0; i<n; i++) {
    obj[i] = blobs[i].centroid;  //Fill obj array
}

I initially noticed that movement was only being tracked in one direction in the x-axis and y-axis because of negative values. I resolved this by changing the calculation for my optical flow by calling the abs() function in cv::Mat.

Mat img1( gray1.getCvImage() );  //Create OpenCV images
Mat img2( gray2.getCvImage() );
Mat flow;
calcOpticalFlowFarneback( img1, img2, flow, 0.7, 3, 11, 5, 5, 1.1, 0 );
//Split flow into separate images
vector<Mat> flowPlanes;
Mat newFlow;
newFlow = abs(flow); //abs flow so values are absolute. Allows tracking in both directions.
split( newFlow, flowPlanes );
//Copy float planes to ofxCv images flowX and flowY
IplImage iplX( flowPlanes[0] );
flowX = &iplX;
IplImage iplY( flowPlanes[1] );
flowY = &iplY;

来源：https://stackoverflow.com/questions/42176120/creating-a-bounding-box-around-a-field-of-optical-flow-paths

标签

c++

OpenCV

openframeworks

opticalflow