Easiest way for offscreen rendering with QOpenGLWidget

Question

I have a hidden QOpenGLWidget (Qt 5.4.2, NOT QGLWidget) and I want to basically continually do grab() or grabFramebuffer() to get its content (and write it to disk). The wid

Answer 1

For offscreen rendering we can use also QOffscreenSurface.

Here is worked class based on Bim's MyGLWindow (not all is tested):

Header OpenGlOffscreenSurface.h:

#ifndef OPENGLOFFSCREENSURFACE_H
#define OPENGLOFFSCREENSURFACE_H

#pragma once

#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 

#include 
#include 

class OpenGlOffscreenSurface
    : public QOffscreenSurface
{
    Q_OBJECT

public:
    /// @brief Constructor. Creates a render window.
    /// @param targetScreen Target screen.
    /// @param size Initial size of a surface buffer.
    /// this is because before the FBO and off-screen surface haven't been created.
    /// By default this uses the QWindow::requestedFormat() for OpenGL context and off-screen
    /// surface.
    explicit OpenGlOffscreenSurface(
            QScreen* targetScreen = nullptr,
            const QSize& size = QSize (1, 1));

    /// @brief Destructor.
    virtual ~OpenGlOffscreenSurface();

    /// @brief Check if the window is initialized and can be used for rendering.
    /// @return Returns true if context, surface and FBO have been set up to start rendering.
    bool isValid() const;

    /// @brief Return the context used in this window.
    /// @return The context used in this window or nullptr if it hasn't been created yet.
    QOpenGLContext* context() const;

    /// @brief Return the OpenGL function object that can be used the issue OpenGL commands.
    /// @return The functions for the context or nullptr if it the context hasn't been created yet.
    QOpenGLFunctions* functions() const;

    /// @brief Return the OpenGL off-screen frame buffer object identifier.
    /// @return The OpenGL off-screen frame buffer object identifier or 0 if no FBO has been created
    /// yet.
    /// @note This changes on every resize!
    GLuint framebufferObjectHandle() const;

    /// @brief Return the OpenGL off-screen frame buffer object.
    /// @return The OpenGL off-screen frame buffer object or nullptr if no FBO has been created yet.
    /// @note This changes on every resize!
    const QOpenGLFramebufferObject* getFramebufferObject() const;

    /// @brief Return the QPaintDevice for paint into it.
    QPaintDevice* getPaintDevice() const;

    /// @brief Return the OpenGL off-screen frame buffer object identifier.
    /// @return The OpenGL off-screen frame buffer object identifier or 0 if no FBO has been created
    /// yet.
    void bindFramebufferObject();

    /// @brief Return the current contents of the FBO.
    /// @return FBO content as 32bit QImage. You might need to swap RGBA to BGRA or vice-versa.
    QImage grabFramebuffer();

    /// @brief Makes the OpenGL context current for rendering.
    /// @note Make sure to bindFramebufferObject() if you want to render to this widgets FBO.
    void makeCurrent();

    /// @brief Release the OpenGL context.
    void doneCurrent();

    /// @brief Copy content of framebuffer to back buffer and swap buffers if the surface is
    /// double-buffered.
    /// If the surface is not double-buffered, the frame buffer content is blitted to the front
    /// buffer.
    /// If the window is not exposed, only the OpenGL pipeline is glFlush()ed so the framebuffer can
    /// be read back.
    void swapBuffers();

    /// @brief Use bufferSize() instead size() for get a size of a surface buffer. We can't override size() as it is not virtual.
    QSize bufferSize() const;

    /// @brief Resize surface buffer to newSize.
    void resize(const QSize& newSize);

    /// @brief Resize surface buffer to size with width w and height h.
    /// @param w Width.
    /// @param h Height.
    void resize(int w, int h);

public slots:
    /// @brief Lazy update routine like QWidget::update().
    void update();

    /// @brief Immediately render the widget contents to framebuffer.
    void render();

signals:
    /// @brief Emitted when swapBuffers() was called and bufferswapping is done.
    void frameSwapped();

    /// @brief Emitted after a resizeEvent().
    void resized();

protected:
    virtual void exposeEvent(QExposeEvent* e);
    virtual void resizeEvent(QResizeEvent* e);
    virtual bool event(QEvent* e) override;

//    virtual int metric(QPaintDevice::PaintDeviceMetric metric) const override;

    /// @brief Called exactly once when the window is first exposed OR render() is called when the
    /// widget is invisible.
    /// @note After this the off-screen surface and FBO are available.
    virtual void initializeGL() = 0;

    /// @brief Called whenever the window size changes.
    /// @param width New window width.
    /// @param height New window height.
    virtual void resizeGL(
            int width,
            int height) = 0;

    /// @brief Called whenever the window needs to repaint itself. Override to draw OpenGL content.
    /// When this function is called, the context is already current and the correct framebuffer is
    /// bound.
    virtual void paintGL() = 0;

    //      /// @brief Called whenever the window needs to repaint itself. Override to draw QPainter
    // content.
    //      /// @brief This is called AFTER paintGL()! Only needed when painting using a QPainter.
    //      virtual void paintEvent(QPainter & painter) = 0;

private:
    Q_DISABLE_COPY(OpenGlOffscreenSurface)
    /// @brief Initialize the window.
    void initializeInternal();

    /// @brief Internal method that does the actual swap work, NOT using a mutex.
    void swapBuffersInternal();

    /// @brief Internal method that checks state and makes the context current, NOT using a mutex.
    void makeCurrentInternal();

    /// @brief Internal method to grab content of a specific framebuffer.
    QImage grabFramebufferInternal(QOpenGLFramebufferObject* fbo);

    /// @brief (Re-)allocate FBO and paint device if needed due to size changes etc.
    void recreateFBOAndPaintDevice();

    /// @brief False before the window was first exposed OR render() was called.
    std::atomic_bool m_initialized;
    /// @brief False before the overridden initializeGL() was first called.
    bool m_initializedGL = false;
    /// @brief True when currently a window update is pending.
    std::atomic_bool m_updatePending;
    /// @brief Mutex making sure not grabbing while drawing etc.
    std::mutex m_mutex;

    /// @brief OpenGL render context.
    QOpenGLContext* m_context = nullptr;
    /// @brief The OpenGL 2.1 / ES 2.0 function object that can be used the issue OpenGL commands.
    QOpenGLFunctions* m_functions = nullptr;
    /// @brief The OpenGL 3.0 function object that can be used the issue OpenGL commands.
    QOpenGLFunctions_3_0* m_functions_3_0 = nullptr;
    /// @brief OpenGL paint device for painting with a QPainter.
    QOpenGLPaintDevice* m_paintDevice = nullptr;
    /// @brief Background FBO for off-screen rendering when the window is not exposed.
    QOpenGLFramebufferObject* m_fbo = nullptr;
    /// @brief Background FBO resolving a multi sampling frame buffer in m_fbo to a frame buffer
    /// that can be grabbed to a QImage.
    QOpenGLFramebufferObject* m_resolvedFbo = nullptr;

    /// @brief Shader used for blitting m_fbo to screen if glBlitFrameBuffer is not available.
    QOpenGLShaderProgram* m_blitShader;

    QSize m_size;
};

#endif  // OPENGLOFFSCREENSURFACE_H

Source OpenGlOffscreenSurface.cpp:

#include "OpenGlOffscreenSurface.h"

#include 
#include 

OpenGlOffscreenSurface::OpenGlOffscreenSurface(
        QScreen*     targetScreen,
        const QSize& size)
    : QOffscreenSurface(targetScreen)
    , m_size(size)
{
    setFormat(QSurfaceFormat::defaultFormat());
    m_initialized = false;
    m_updatePending = false;
    create();  // Some platforms require this function to be called on the main (GUI) thread
    initializeInternal();
}


OpenGlOffscreenSurface::~OpenGlOffscreenSurface()
{
    // to delete the FBOs we first need to make the context current
    m_context->makeCurrent(this);
    // destroy framebuffer objects
    if (m_fbo) {
        m_fbo->release();
        delete m_fbo;
        m_fbo = nullptr;
    }
    if (m_resolvedFbo) {
        m_resolvedFbo->release();
        delete m_resolvedFbo;
        m_resolvedFbo = nullptr;
    }
    // destroy shader
    delete m_blitShader;
    m_blitShader = nullptr;
    // free context
    m_context->doneCurrent();
    delete m_context;
    m_context = nullptr;
    // free paint device
    delete m_paintDevice;
    m_paintDevice = nullptr;
    m_initialized = false;
    m_updatePending = false;
    destroy();
}


QOpenGLContext* OpenGlOffscreenSurface::context() const
{
    return (m_context);
}


QOpenGLFunctions* OpenGlOffscreenSurface::functions() const
{
    return (m_functions);
}


GLuint OpenGlOffscreenSurface::framebufferObjectHandle() const
{
    return (m_fbo ? m_fbo->handle() : 0);
}


const QOpenGLFramebufferObject* OpenGlOffscreenSurface::getFramebufferObject() const
{
    return (m_fbo);
}


QPaintDevice* OpenGlOffscreenSurface::getPaintDevice() const
{
    return (m_paintDevice);
}


void OpenGlOffscreenSurface::bindFramebufferObject()
{
    if (m_fbo) {
        m_fbo->bind();
    } else {
        QOpenGLFramebufferObject::bindDefault();
    }
}


bool OpenGlOffscreenSurface::isValid() const
{
    return (m_initialized && m_context && m_fbo);
}


void OpenGlOffscreenSurface::makeCurrent()
{
    makeCurrentInternal();
}


void OpenGlOffscreenSurface::makeCurrentInternal()
{
    if (isValid()) {
        m_context->makeCurrent(this);
    } else {
        throw ("OpenGlOffscreenSurface::makeCurrent() - Window not yet properly initialized!");
    }
}


void OpenGlOffscreenSurface::doneCurrent()
{
    if (m_context) {
        m_context->doneCurrent();
    }
}


QImage OpenGlOffscreenSurface::grabFramebuffer()
{
    std::lock_guard  locker(m_mutex);
    makeCurrentInternal();
    // blit framebuffer to resolve framebuffer first if needed
    if (m_fbo->format().samples() > 0) {
        // check if we have glFrameBufferBlit support. this is true for desktop OpenGL 3.0+, but not
        // OpenGL ES 2.0
        if (m_functions_3_0) {
            // only blit the color buffer attachment
            m_functions_3_0->glBindFramebuffer(GL_READ_FRAMEBUFFER, m_fbo->handle());
            m_functions_3_0->glBindFramebuffer(GL_DRAW_FRAMEBUFFER, m_resolvedFbo->handle());
            m_functions_3_0->glBlitFramebuffer(0, 0, bufferSize().width(),
                    bufferSize().height(), 0, 0, bufferSize().width(),
                    bufferSize().height(), GL_COLOR_BUFFER_BIT, GL_NEAREST);
            m_functions_3_0->glBindFramebuffer(GL_FRAMEBUFFER, 0);
        } else {
            // we must unbind the FBO here, so we can use its texture and bind the default
            // back-buffer
            m_functions->glBindFramebuffer(GL_FRAMEBUFFER, m_resolvedFbo->handle());
            // now use its texture for drawing in the shader
            // --> bind shader and draw textured quad here
            // bind regular FBO again
            m_functions->glBindFramebuffer(GL_FRAMEBUFFER, m_fbo->handle());
        }
        // check if OpenGL errors happened
        if (GLenum error = m_functions->glGetError() != GL_NO_ERROR) {
            qDebug() << "OpenGlOffscreenSurface::grabFramebuffer() - OpenGL error" << error;
        }

        // now grab from resolve FBO
        return (grabFramebufferInternal(m_resolvedFbo));
    } else {
        // no multi-sampling. grab directly from FBO
        return (grabFramebufferInternal(m_fbo));
    }
}  // OpenGlOffscreenSurface::grabFramebuffer


QImage OpenGlOffscreenSurface::grabFramebufferInternal(QOpenGLFramebufferObject* fbo)
{
    QImage image;
    // bind framebuffer first
    m_functions->glBindFramebuffer(GL_READ_FRAMEBUFFER, fbo->handle());
    if (m_functions_3_0) {
        m_functions_3_0->glReadBuffer(GL_COLOR_ATTACHMENT0);
    }
    GLenum internalFormat = fbo->format().internalTextureFormat();
    bool hasAlpha = internalFormat == GL_RGBA || internalFormat == GL_BGRA
                    || internalFormat == GL_RGBA8;
    if (internalFormat == GL_BGRA) {
        image = QImage(fbo->size(), hasAlpha ? QImage::Format_ARGB32 : QImage::Format_RGB32);
        m_functions->glReadPixels(0, 0, fbo->size().width(),
                fbo->size().height(), GL_BGRA, GL_UNSIGNED_BYTE, image.bits());
    } else if ((internalFormat == GL_RGBA) || (internalFormat == GL_RGBA8)) {
        image = QImage(fbo->size(), hasAlpha ? QImage::Format_RGBA8888 : QImage::Format_RGBX8888);
        m_functions->glReadPixels(0, 0, fbo->size().width(),
                fbo->size().height(), GL_RGBA, GL_UNSIGNED_BYTE, image.bits());
    } else {
        qDebug() << "OpenGlOffscreenSurface::grabFramebuffer() - Unsupported framebuffer format"
                 << internalFormat << "!";
    }
    m_functions->glBindFramebuffer(GL_FRAMEBUFFER, m_fbo->handle());

    return (image.mirrored());
}  // OpenGlOffscreenSurface::grabFramebufferInternal


void OpenGlOffscreenSurface::swapBuffers()
{
    swapBuffersInternal();
    emit frameSwapped();
}


void OpenGlOffscreenSurface::swapBuffersInternal()
{
    // blit framebuffer to back buffer
    m_context->makeCurrent(this);
    // make sure all paint operation have been processed
    m_functions->glFlush();
    // check if we have glFrameBufferBlit support. this is true for desktop OpenGL 3.0+, but not
    // OpenGL ES 2.0
    if (m_functions_3_0) {
        // if our framebuffer has multi-sampling, the resolve should be done automagically
        m_functions_3_0->glBindFramebuffer(GL_READ_FRAMEBUFFER, m_fbo->handle());
        m_functions_3_0->glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0);
        // blit all buffers including depth buffer for further rendering
        m_functions_3_0->glBlitFramebuffer(0, 0, bufferSize().width(),
                bufferSize().height(), 0, 0, bufferSize().width(),
                bufferSize().height(), GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT,
                GL_NEAREST);
        m_functions_3_0->glBindFramebuffer(GL_FRAMEBUFFER, m_fbo->handle());
    } else {
        // we must unbind the FBO here, so we can use its texture and bind the default back-buffer
        m_functions->glBindFramebuffer(GL_FRAMEBUFFER, 0);
        // now use its texture for drawing in the shader
        // --> bind shader and draw textured quad here
        // bind regular FBO again
        m_functions->glBindFramebuffer(GL_FRAMEBUFFER, m_fbo->handle());
    }
    // check if OpenGL errors happened
    if (GLenum error = m_functions->glGetError() != GL_NO_ERROR) {
        qDebug() << "OpenGlOffscreenSurface::swapBuffersInternal() - OpenGL error" << error;
    }
    // now swap back buffer to front buffer
    m_context->swapBuffers(this);
}  // OpenGlOffscreenSurface::swapBuffersInternal


void OpenGlOffscreenSurface::recreateFBOAndPaintDevice()
{
    if (m_context && ((m_fbo == nullptr) || (m_fbo->size() != bufferSize()))) {
        m_context->makeCurrent(this);
        // free old FBOs
        if (m_fbo) {
            m_fbo->release();
            delete m_fbo;
            m_fbo = nullptr;
        }
        if (m_resolvedFbo) {
            m_resolvedFbo->release();
            delete m_resolvedFbo;
            m_resolvedFbo = nullptr;
        }
        // create new frame buffer
//        QOpenGLFramebufferObjectFormat format = QGLInfo::DefaultFramebufferFormat();
//        format.setSamples(QGLInfo::HasMultisamplingSupport(m_context) ? 4 : 0);
        QOpenGLFramebufferObjectFormat format;
        format.setSamples(0);

        m_fbo = new QOpenGLFramebufferObject(bufferSize(), format);
        if (!m_fbo->isValid()) {
            throw ("OpenGlOffscreenSurface::recreateFbo() - Failed to create background FBO!");
        }
        // clear framebuffer
        m_fbo->bind();
        m_functions->glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
        m_fbo->release();
        // if multi sampling is requested and supported we need a resolve FBO
        if (format.samples() > 0) {
            // create resolve framebuffer with only a color attachment
            format.setAttachment(QOpenGLFramebufferObject::NoAttachment);
            format.setSamples(0);
            m_resolvedFbo = new QOpenGLFramebufferObject(bufferSize(), format);
            if (!m_resolvedFbo->isValid()) {
                throw ("OpenGlOffscreenSurface::recreateFbo() - Failed to create resolve FBO!");
            }
            // clear resolve framebuffer
            m_resolvedFbo->bind();
            m_functions->glClear(GL_COLOR_BUFFER_BIT);
            m_resolvedFbo->release();
        }
    }
    // create paint device for painting with QPainter if needed
    if (!m_paintDevice) {
        m_paintDevice = new QOpenGLPaintDevice;
    }
    // update paint device size if needed
    if (m_paintDevice->size() != bufferSize()) {
        m_paintDevice->setSize(bufferSize());
    }
}  // OpenGlOffscreenSurface::recreateFBOAndPaintDevice


void OpenGlOffscreenSurface::initializeInternal()
{
    if (!m_initialized.exchange(true)) {
        // create OpenGL context. we set the format requested by the user (default:
        // QWindow::requestedFormat())
        m_context = new QOpenGLContext(this);
        m_context->setFormat(format());
        if (m_context->create()) {
            m_context->makeCurrent(this);
            // initialize the OpenGL 2.1 / ES 2.0 functions for this object
            m_functions = m_context->functions();
            m_functions->initializeOpenGLFunctions();
            // try initializing the OpenGL 3.0 functions for this object
            m_functions_3_0 = m_context->versionFunctions ();
            if (m_functions_3_0) {
                m_functions_3_0->initializeOpenGLFunctions();
            } else {
                // if we do not have OpenGL 3.0 functions, glBlitFrameBuffer is not available, so we
                // must do the blit
                // using a shader and the framebuffer texture, so we need to create the shader
                // here...
                // --> allocate m_blitShader, a simple shader for drawing a textured quad
                // --> build quad geometry, VBO, whatever
            }
            // now we have a context, create the FBO
            recreateFBOAndPaintDevice();
        } else {
            m_initialized = false;
            delete m_context;
            m_context = nullptr;
            throw ("Failed to create OpenGL context!");
        }
    }
}  // OpenGlOffscreenSurface::initializeInternal


void OpenGlOffscreenSurface::update()
{
    // only queue an update if there's not already an update pending
    if (!m_updatePending.exchange(true)) {
        QCoreApplication::postEvent(this, new QEvent(QEvent::UpdateRequest));
    }
}


void OpenGlOffscreenSurface::render()
{
    std::lock_guard  locker(m_mutex);
    // check if we need to initialize stuff
    initializeInternal();
    // check if we need to call the user initialization
//    makeCurrent(); // TODO: may be makeCurrent() must be here, as noted for QOpenGLWidget.initializeGL()
    if (!m_initializedGL) {
        m_initializedGL = true;
        initializeGL();
    }
    // make context current and bind framebuffer
    makeCurrent();
    bindFramebufferObject();
    // call user paint function
    paintGL();
    doneCurrent();
    // mark that we're done with updating
    m_updatePending = false;
}  // OpenGlOffscreenSurface::render


void OpenGlOffscreenSurface::exposeEvent(QExposeEvent* e)
{
    // render window content if window is exposed
    render();
}  // OpenGlOffscreenSurface::exposeEvent


void OpenGlOffscreenSurface::resizeEvent(QResizeEvent* e)
{
    // call base implementation
    resize(e->size());
    emit resized();
}


void OpenGlOffscreenSurface::resize(const QSize& newSize)
{
    m_mutex.lock();
    // make context current first
    makeCurrent();

    m_size = QSize(newSize);

    // update FBO and paint device
    recreateFBOAndPaintDevice();
    m_mutex.unlock();
    // call user-defined resize method
    resizeGL(bufferSize().width(), bufferSize().height());
}  // OpenGlOffscreenSurface::resize


void OpenGlOffscreenSurface::resize(
        int w,
        int h)
{
    resize(QSize(w, h));
}


bool OpenGlOffscreenSurface::event(QEvent* event)
{
    switch (event->type()) {
        case QEvent::UpdateLater:
            update();
            return (true);

        case QEvent::UpdateRequest:
            render();
            return (true);

        default:
            return (false);
    }  // switch
}  // OpenGlOffscreenSurface::event


QSize OpenGlOffscreenSurface::bufferSize() const
{
    return (m_size);
}

Using OpenGlOffscreenSurface:

Header ExamplePaintSurface.h:

#ifndef EXAMPLEPAINTSURFACE_H
#define EXAMPLEPAINTSURFACE_H

#include "OpenGlOffscreenSurface.h"

class ExamplePaintSurface
    : public OpenGlOffscreenSurface
{
public:
    explicit ExamplePaintSurface(
            QScreen* targetScreen = nullptr,
            const QSize& size = QSize (1, 1));

    virtual ~ExamplePaintSurface() override;

protected:
    virtual void initializeGL() override;

    virtual void resizeGL(
            int width,
            int height) override;

    virtual void paintGL() override;
};


#endif  // EXAMPLEPAINTSURFACE_H

Source ExamplePaintSurface.cpp:

#include "ExamplePaintSurface.h"

#include 

ExamplePaintSurface::ExamplePaintSurface(
        QScreen*     targetScreen,
        const QSize& size)
    : OpenGlOffscreenSurface(targetScreen, size) {}


ExamplePaintSurface::~ExamplePaintSurface() {}


void ExamplePaintSurface::initializeGL() {}


void ExamplePaintSurface::resizeGL(int width, int height) {}


void ExamplePaintSurface::paintGL()
{
    QPainter painter(getPaintDevice());
    painter.setRenderHints(QPainter::Antialiasing | QPainter::TextAntialiasing);

    painter.drawText(20, 40, "Test"); // <-- drawing here

    painter.end();
}

Source main.cpp:

#include 

#include "ExamplePaintSurface.h"


int main(int argc, char *argv[])
{
    QApplication a(argc, argv);

    ExamplePaintSurface paintSurface;
    paintSurface.resize(300, 200);
    paintSurface.render();
    QImage image = paintSurface.grabFramebuffer();
    image.save(QString("image.png"));

    return a.exec();
}

Part example with QOffscreenSurface is from here:

#include 
#include 
#include 
#include 
#include 
#include 

int main(int argc, char* argv[])
{
   QGuiApplication a(argc, argv);

   QSurfaceFormat surfaceFormat;
   surfaceFormat.setMajorVersion(4);
   surfaceFormat.setMinorVersion(3);

   QOpenGLContext openGLContext;
   openGLContext.setFormat(surfaceFormat);
   openGLContext.create();
   if(!openGLContext.isValid()) return -1;

   QOffscreenSurface surface;
   surface.setFormat(surfaceFormat);
   surface.create();
   if(!surface.isValid()) return -2;

   openGLContext.makeCurrent(&surface);

   QOpenGLFunctions_4_3_Core f;
   if(!f.initializeOpenGLFunctions()) return -3;

   qDebug() << QString::fromLatin1((const char*)f.glGetString(GL_VERSION));

   return 0;
}

And here is an aswer about it:

For offscreen rendering, try rendering into a QOpenGLFramebufferObject, which can be converted into a QImage, which in turn can easily be saved to disk.

For that however, you still need a surface to render onto (as required by QOpenGLContext::makeCurrent()), so your only choice is indeed using a QWindow or a QGLWidget to get such a surface.

In Qt 5.1, there will be a new class called QOffscreenSurface, which will probably be most suitable for your usecase. You would use QOffscreenSurface to get a surface for your OpenGL context, and then render into a FBO using QOpenGLFramebufferObject, and then call QOpenGLFramebufferObject::toImage() to get access to the pixels.