OpenCV VideoWriter Framerate Issue

Question

I'm attempting to record video from a 1080p webcam into a file. I held a timer up in the video and in every trial the timestamp reported by a video player (VLC is what I used) doesn't sync up with the time in the video. It's always off a few seconds (usually in-video timer is faster than player-reported time).

As seen below, I set up the C++ program to capture video in one thread, and record in another thread. This is working fine as my CPU usage is ~200% (possible max out?). I'm on a Macbook Air w/ OS X 10.8 @ 1.8 GHz Intel Core i7.

I've tried changing the framerate to 15fps and that results in very choppy/slow video. I've also tried setting CV_CAP_PROP_FRAME_WIDTH & CV_CAP_PROP_FRAME_HEIGHT to a lower resolution and it results in slow video. It appears that 1080p @ 30fps results in good steady video, but it is still always plays faster than it's supposed to. I've also tried putting in waitKey(10); after record << frame; but it did not affect anything.

Any recommendations on how to make the video match up in time?

Thanks! Aakash

#include "opencv/cv.h"
#include "opencv/highgui.h"
#include <boost/thread.hpp>

using namespace cv;

void captureFunc(Mat *frame, VideoCapture *capture){
    for(;;){
        // get a new frame from camera
        (*capture) >> (*frame);
    }
}

int main(int, char**)
{
    VideoCapture capture(0); // open the default camera
    if( !capture.isOpened() )  {
        printf("Camera failed to open!\n");
        return -1;
    }

    capture.set(CV_CAP_PROP_FPS,30); //set capture rate to 30fps
    Mat frame;
    capture >> frame; // get first frame for size

    // initialize recording of video
    VideoWriter record("test.avi", CV_FOURCC('D','I','V','X'), 30, frame.size(), true);
    if( !record.isOpened() ) {
        printf("VideoWriter failed to open!\n");
        return -1;
    }

    boost::thread captureThread(captureFunc, &frame, &capture); //start capture thread

    sleep(1); //just to make sure capture thread is ready

    for(;;)
    {
        // add frame to recorded video
        record << frame;
    }

    return 0;
}

Answer 1

I resolved my issue after a bit of debugging; it was an issue with VideoWriter being picky on the rate at which frames were fed to it.

Answer 2

You need to sync your read and write functions. Your code reads as fast as possible, and writes also as fast as possible. Your output video probably looks slow because writing the output happens faster than reading the input (since capture >> is waiting for your camera), and several identical frames are recorded.

Writing without waiting or syncing means you may write the same content several times (what I think is happening here), or lose frames.

If you want to keep having threads, you may, but you will need to make your write process wait until there is something new to write.

Likewise to avoid losing frames or writing corrupted frames, you need the read process to wait until writing is done, so that frame can be safely overwritten.

Since the threads need to wait for each other anyway, there's little point in threads at all.

I'd rather recommend this much simpler way:

for (;;) {
    capture >> frame;
    process(frame);  // whatever smart you need
    record << frame;
}

If you need parallelism, you'll need much more complex sync mechanism, and maybe some kind of fifo for your frames.