New Qt6 QML (OpenGL) elements

Recently, I was tasked with porting some of the existing Qt5-based QML GStreamer elements for use with Qt6. Throughout this process, the only major architectural change in Qt that affected these GStreamer elements was the Qt6 rendering overhaul for supporting multiple graphics APIs (OpenGL, Vulkan, Direct3D, Metal, etc). The process overall was relatively painless however still required understanding the new Qt6 rendering model. The result is a set of elements that mirror functionality of the existing Qt5 GStreamer elements (`qmlglsink`, `qmlglsrc`, `qmlgloverlay`). The new elements are named `qml6glsink`, `qml6glsrc`, and `qml6gloverlay`. There is also a new element called `qml6glmixer`.

qml6glsink

The Qt6 variant of qmlglsink operates the exact same as its Qt5 counterpart. There is a QML Item that needs to be retrieved from the QML scene and a GStreamer sink component that needs the QML item set as a property. Once that is achieved, the video stream is viewable in the QML scene just like the Qt5 counterpart. You can a Qt6 example of this in this qmlsink example.

qml6glsrc

The qt6 variant of qmlglsrc operates the same as its Qt5 counterpart. The required interface in this case is a `QQuickWindow *`. When QML draws to this QQuickWindow, with some careful overriding of some Qt6 signals, the output is rendered to a GStreamer controlled texture that is then pushed downstream to the next OpenGL element. An example of this can be found in this qmlsrc example.

qml6gloverlay

The Qt6 variant of qmlgloverlay operates the exact same as its Qt5 counterpart. With the qml6gloverlay GStreamer element, a QML scene is drawn including the incoming video stream. The resulting texture is then pushed downstream for further processing. The connection between the QML scene and the GStreamer input frames is also provided by a special QML item like in the qml6glsink case. An example of this can be found in this qmloverlay example.

qml6glmixer - New Element

The qml6glmixer element does not have a Qt5 counterpart. It functions much like the qml6gloverlay element, however it can source images from multiple input video streams. The application is required to connect the relevant QML item to the correct input GStreamer pad on the qml6glmixer element. The input videos can then be controlled from the QML scene in many different ways (e.g. position, fades, shaders, etc). A simple example of qml6glmixer can be found in this qmlmixer example.

I hope these elements are useful for you and I can't wait to see what you make with them.

Comments

Post a Comment

Popular posts from this blog

GStreamer RTP Session Handling in Rust

Recently we at Centricular had the opportunity to write some new GStreamer RTP elements in Rust as part of a grant from the Sovereign Tech Fund . We of course jumped at the chance. The existing RTP elements are all currently written in the C programming language and have been around for over a decade. The age is showing somewhat with some of the architecture decisions and user visible interfaces. There is also a general desire within the GStreamer community to write parsers and other elements handling potentially untrusted data in a much more memory safe language, like Rust. As a bonus, Rust comes with better tooling out of the box for developers than C/C++. In this blog post we will be looking into how RTP sessions are managed in GStreamer, independent of the specifics of a particular RTP payload format. The Prior Art: rtpbin In GStreamer, the pre-existing interface for handling RTP sessions is managed by the rtpbin element. The rtpbin element is over 15 years old and ha...
Read more

GStreamer 1.6 and OpenGL contexts

With the imminent release of GStreamer 1.6 there has been a change in the mechanism by which OpenGL contexts and Window System (X11, Wayland, Win32, OS X's Cocoa) specific resources are shared with a GStreamer pipeline for elements that need that information. Elements like sinks or any of the GL elements or any global application provided information that is shared between elements. Previously, in 1.4, there was a property, 'external-opengl-context' available on some(!) elements that one could set on the current set of filter and mixer elements (not sink or source elements though!). While it mostly worked well enough, it was a behaviour difference to how window system handles are shared between the application and the GStreamer pipeline. Internal GL context sharing between elements was occurring in the ALLOCATION query as a value in a GstStructure of the meta:GstVideoGLTextureUploadMeta parameters. The main problem with this is 1. the ALLOCATION query is handled diffe...
Read more

qmlglsink - GStreamer and Qt's QML

Windows Support - NEW!! Just recently, I added the missing piece for adding Windows support for qmlglsink for QtQuick 2 applications (tracked by this bug ) with this commit which means that qmlglsink is now usable on all major platforms 1 . With the list of supported platforms (Android, iOS, OS X, Windows, Linux) for qmlglsink growing, I thought I'd write a little bit about it and why you should use it. qmlglsink is a GStreamer element that will render a video stream into a QtQuick QML scene. As you can see from this example , qmlglsink does this by providing a GstGLVideoItem QML item that can be placed anywhere in the scene. The GstGLVideoItem is given a unique name to be able to link the element and the QML item. Without this link, the element cannot render the stream anywhere and will fail changing state. In the C++ code, we then retrieve the GstGLVideoItem by name like so . Then we set the pipeline to playing, start Qt's mainloop and watch the show. Why Use qml...
Read more