Optical flow based segmentation

Question

Hi I would like to segment objects in a video sequence based on optical flow (as suggested in this answer), I am using opencv and have been able to generate a flow field for

Answer 1

One very straightforward way to go about it will be to provide the algorithm with a key frame which has the ground truths. What it means is that, segment the first frame manually (you could use a grabcut here) and then propagate the segmentation to subsequent frames using optical flow.

The caveat is that optical flow is notoriously famous for being error prone. Hence instead of using the propagated segmentation directly, you can erode (using some structuring element) your propagated segmentation and use the eroded image as a foreground seed for grabcut. Similarly, dilate the propagated segementation and use a band around the dilated boundary as a background seed.

In short: For the first frame give the ground truth, use that to automatically perform grabcut on subsequent frames.

Does this help?