Opening a GStreamer pipeline from OpenCV with VideoWriter

Question

I am capturing and processing video frames with OpenCV, and I would like to write them as a h265 video file. I am struggling to get a proper Gstreamer pipeline to work from OpenCV.

Gstreamer works fine by itself. In particular, I am able to run this command, which encodes video very quickly (thanks to GPU acceleration) and saves it to a mkv file:

gst-launch-1.0 videotestsrc num-buffers=90 ! 'video/x-raw, format=(string)I420, width=(int)640, height=(int)480' ! omxh265enc ! matroskamux ! filesink location=test.mkv

Now I would like to do the same thing from within my OpenCV application. My code is something like:

Mat img_vid = Mat(1024, 1024, CV_8UC3);

VideoWriter video;
video.open("appsrc ! autovideoconvert ! omxh265enc ! matroskamux ! filesink location=test.mkv", 0, (double)25, cv::Size(1024, 1024), true);

if (!video.isOpened()) {
   printf("can't create writer\n");
   return -1;
}

while ( ... ) {

   // Capture frame into img_vid => That works fine

   video.write(img_vid);

   ...
}

At first sight, this seems to work, but what it does is it creates file named "appsrc ! autovideoconvert ! omxh265enc ! matroskamux ! filesink location=test.mkv" and fills it with uncompressed video frames, completely ignoring the fact that this is a Gstreamer pipeline.

I have tried other pipelines, but they result in a variety of errors:

video.open("appsrc ! autovideoconvert ! omxh264enc ! 'video/x-h264, streamformat=(string)byte-stream' ! h264parse ! qtmux ! filesink location=test.mp4 -e", 0, (double)25, cv::Size(1024, 1024), true);

Which results in:

(Test:5533): GStreamer-CRITICAL **: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed OpenCV Error: Unspecified error (GStreamer: cannot find appsrc in manual pipeline ) in CvVideoWriter_GStreamer::open, file /home/ubuntu/opencv/modules/videoio/src/cap_gstreamer.cpp, line 1363 VIDEOIO(cvCreateVideoWriter_GStreamer(filename, fourcc, fps, frameSize, is_color)): raised OpenCV exception:

/home/ubuntu/opencv/modules/videoio/src/cap_gstreamer.cpp:1363: error: (-2) GStreamer: cannot find appsrc in manual pipeline in function CvVideoWriter_GStreamer::open

I also tried the simple:

video.open("appsrc ! autovideosink", 0, (double)25, cv::Size(1024, 1024), true);

which yields:

GStreamer Plugin: Embedded video playback halted; module appsrc0 reported: Internal data flow error.

I am using OpenCV 3.1 with Gstreamer support. The hardware is a Jetson TX1 with L4T 24.2.1.

Answer 1

I encountered a similar problem before. Since the pipe/file name ends with .mkv, OpenCV interprets it as a video file instead of a pipe.

You can try ending it with a dummy spacing like after mkv

video.open("appsrc ! autovideoconvert ! omxh265enc ! matroskamux ! filesink location=test.mkv ", 0, (double)25, cv::Size(1024, 1024), true);

or with a dummy property like

video.open("appsrc ! autovideoconvert ! omxh265enc ! matroskamux ! filesink location=test.mkv sync=false", 0, (double)25, cv::Size(1024, 1024), true);