qmlglsink - GStreamer and Qt's QML

Windows Support - NEW!!

Just recently, I added the missing piece for adding Windows support for qmlglsink for QtQuick 2 applications (tracked by this bug) with this commit which means that qmlglsink is now usable on all major platforms1. With the list of supported platforms (Android, iOS, OS X, Windows, Linux) for qmlglsink growing, I thought I'd write a little bit about it and why you should use it.

qmlglsink is a GStreamer element that will render a video stream into a QtQuick QML scene. As you can see from this example, qmlglsink does this by providing a GstGLVideoItem QML item that can be placed anywhere in the scene. The GstGLVideoItem is given a unique name to be able to link the element and the QML item. Without this link, the element cannot render the stream anywhere and will fail changing state. In the C++ code, we then retrieve the GstGLVideoItem by name like so. Then we set the pipeline to playing, start Qt's mainloop and watch the show.

Why Use qmlglsink?

The main reason that you should have a look at qmlglsink is if you need the full flexibility of GStreamer across multiple platforms. QtMultimedia only provides a small subset of the features possible with GStreamer and only uses GStreamer on unix systems. Only file-based playback or Buffer-based playback (depending on the backend) is supported by QtMultimedia and it does not come by default with zerocopy support for any platform.

GStreamer on the other hand supports a multitude of media sources from network streaming like rtp and rtsp, file and memory sources as well as recent GStreamer versions containing zerocopy support for Android, iOS2, the RPi, and some support for zerocopy on desktop Linux with VA-API. GStreamer is also extremely flexible and extensible as a multitude of different multimedia tasks are just a plugin away.

1 - Not all platforms are easy to build against Qt. Notably the recent Windows support requires manual intervention with qmake utilizing the added qtplugin.pro file. iOS has been tested against specially modified Qt pkg-config and libtool files.
2 - Unfortunately the iOS zerocopy support is not usable with Qt due to slight incompatibilities between OpenGL|ES 2 and OpenGL|ES 3 on the iOS platform. Qt currently defaults to OpenGL|ES 3 while GStreamer is still choosing OpenGL|ES 2 for zerocopy support. On all other platforms OpenGL ES 2 is compatible with OpenGL ES 3. Edit: This is no longer an issue with this and this commit

Comments

  1. amorenoc16 July 2016 at 03:42

    nice work!

    ReplyDelete
  2. Pieter6 August 2019 at 04:00

    For android, do you simply link the libraries built for arm? Or how does the development/deployment work for android + qmlglsink (and other gstreamer elements)?

    ReplyDelete

Post a Comment

Popular posts from this blog

GStreamer RTP Session Handling in Rust

Recently we at Centricular had the opportunity to write some new GStreamer RTP elements in Rust as part of a grant from the Sovereign Tech Fund . We of course jumped at the chance. The existing RTP elements are all currently written in the C programming language and have been around for over a decade. The age is showing somewhat with some of the architecture decisions and user visible interfaces. There is also a general desire within the GStreamer community to write parsers and other elements handling potentially untrusted data in a much more memory safe language, like Rust. As a bonus, Rust comes with better tooling out of the box for developers than C/C++. In this blog post we will be looking into how RTP sessions are managed in GStreamer, independent of the specifics of a particular RTP payload format. The Prior Art: rtpbin In GStreamer, the pre-existing interface for handling RTP sessions is managed by the rtpbin element. The rtpbin element is over 15 years old and ha...
Read more

GStreamer 1.6 and OpenGL contexts

With the imminent release of GStreamer 1.6 there has been a change in the mechanism by which OpenGL contexts and Window System (X11, Wayland, Win32, OS X's Cocoa) specific resources are shared with a GStreamer pipeline for elements that need that information. Elements like sinks or any of the GL elements or any global application provided information that is shared between elements. Previously, in 1.4, there was a property, 'external-opengl-context' available on some(!) elements that one could set on the current set of filter and mixer elements (not sink or source elements though!). While it mostly worked well enough, it was a behaviour difference to how window system handles are shared between the application and the GStreamer pipeline. Internal GL context sharing between elements was occurring in the ALLOCATION query as a value in a GstStructure of the meta:GstVideoGLTextureUploadMeta parameters. The main problem with this is 1. the ALLOCATION query is handled diffe...
Read more