Custom source property for VideoOutput QML

Question

What exactly has to be done to provide custom source of frames for VideoOutput QML object?

Does VideoOuput itself provide an instance of QAbstractVideoSurface class to the "source"?

The Qt5 documentations says following thing about providing about this issue:

If you are extending your own C++ classes to interoperate with VideoOutput, you can either provide a QObject based class with a mediaObject property that exposes a QMediaObject derived class that has a QVideoRendererControl available, or you can provide a QObject based class with a writable videoSurface property that can accept a QAbstractVideoSurface based class and can follow the correct protocol to deliver QVideoFrames to it.

In accordance to following documentation I did the following:

1. I implmented my own class myFrameProvider derived from QObject that has writable videoSurface property.
2. Make a class connected to the following that sends frames to myFrameProvider.
3. Instantianted myFrameProvider class and made it accessible in the same QML context as `VideoOutput' widget.

After that - I'm getting the segfault whenever "videSurface" property is accessed. Should I set my own video surface property??

Answer 1

While having a similar problem, I stumbled across your question. A bit after, I found a solution, which worked for me. Even if your question is an older one and you probably moved on, I want to share my answer to potentially help other people.

I found the answer in the QT documentation in the section "Working with Low Level Video Frames". The piece of code posted there, was very helpful as starting point, but I had to modify it, so that it woks properly. A minimal working example looks like this:

FrameProvider.h

#include <QObject>
#include <QAbstractVideoSurface>
#include <QVideoSurfaceFormat>

class FameProvider : public QObject
{
    Q_OBJECT
    Q_PROPERTY(QAbstractVideoSurface *videoSurface READ videoSurface WRITE setVideoSurface)


public:
    QAbstractVideoSurface* videoSurface() const { return m_surface; }

private:
    QAbstractVideoSurface *m_surface = NULL;
    QVideoSurfaceFormat m_format;

public:


    void setVideoSurface(QAbstractVideoSurface *surface)
    {
        if (m_surface && m_surface != surface  && m_surface->isActive()) {
            m_surface->stop();
        }

        m_surface = surface;

        if (m_surface && m_format.isValid())
        {
            m_format = m_surface->nearestFormat(m_format);
            m_surface->start(m_format);

        }
    }

    void setFormat(int width, int heigth, int format)
    {
        QSize size(width, heigth);
        QVideoSurfaceFormat format(size, format);
        m_format = format;

        if (m_surface) 
        {
            if (m_surface->isActive())
            {
                m_surface->stop();
            }
            m_format = m_surface->nearestFormat(m_format);
            m_surface->start(m_format);
        }
    }

public slots:
    void onNewVideoContentReceived(const QVideoFrame &frame)
    {

        if (m_surface)
            m_surface->present(frame);
    }
};

main.qml

import QtQuick 2.9
import QtQuick.Controls 2.2
import QtQuick.Controls.Material 2.2
import QtMultimedia 5.4
import com.yourcompany.FrameProvider 1.0

ApplicationWindow {
    objectName: "mainWindow"
    visible: true
    width: 640
    height: 480

    FrameProvider{
        objectName: "provider"
        id: provider
    }

    VideoOutput {
        id: display
        objectName: "display"
        anchors.top: parent.top
        anchors.bottom: parent.bottom
        width: parent.width
        source: provider
    }
}

main.cpp

#include <QGuiApplication>
#include <QQmlApplicationEngine>

 int main(int argc, char *argv[])
{
    // initialize the qml application engine
    QCoreApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
    QGuiApplication app(argc, argv);
    QQmlApplicationEngine engine;

    //register the custom control to the qml application engine
    qmlRegisterType<FameProvider>("com.yourcompany.FrameProvider", 1, 0, "FrameProvider");

    // start the view
    engine.load(QUrl(QStringLiteral("qrc:/main.qml")));
    if (engine.rootObjects().isEmpty())
    {
    return -1;
    }

    // find your custom control
    QObject *rootObject = engine.rootObjects().first();
    Qobject *display = rootObject->findChild<QObject *>("display");
    auto provider = qvariant_cast<FameProvider *>(display->property("source"));

    // Create your custom frame source class, which inherits from QObject. This source is expected to have the following public fields and signals:
    // - int width
    // - int height
    // - int format (following QVideoFrame::PixelFormat)
    // - signals: void newFrameAvailable(const QVideoFrame &frame);
    CustomFramesource source;

    // Set the correct format for the video surface (Make sure your selected format is supported by the surface)
    provider->setFormat(source.width,source.height, source.format);

    // Connect your frame source with the provider
    QObject::connect(&source, SIGNAL(newFrameAvailable(const QVideoFrame &)), provider, SLOT(onNewVideoContentReceived(const QVideoFrame &)));

    // run the app
    int retVal =  app.exec();

    return 0;
}

The MWE is condensed from my actual code and thus untested. I hope it runs anyway and shows all needed steps.