Hardware accelerated video in 2022

The x86_64 world is a little different to the ARM world. Intel/AMD standardised their hardware decoders around VAAPI so Kodi can call ffmpeg in a consistent way regardless of each, although at some point capabilities need to reflect which GPU/card and what the TV/panel supports. It all works reasonably well though advanced things like tonemapping still need to be plumbed in. Even nVidia finallly caved and added support for GBM/mesa to their latest drivers; and although Kodi does not support NVDEC (needed) someone wrote an NVDEC > VAAPI ship and we have a proof of concept implementation; although you need to run under Wayland which doesn't support fun stuff like "adjust-refresh" by design so it's not quite the solution nVidia users need (better than "no support" though)..

The ARM world is domainated by ARM Mali GPUs (well supported with FOSS drivers now) with separate vendor-specific decoders. These have a mix of "stateless" and "stateful" types under the V4L2 banner. The stateless V4L2 "requests" uAPI is stable and supports a good range of codecs upstream; it's still up to individual drivers to support them. The stateful "mem2mem" side is not declared stable yet, and with the dependency on vendor firmware blobs is generally a bit futher behind. There is a lot of collaboration happening on testing and upstreaming to the kernel - and LE is heavly involved in that (Kodi is a good test app).

On the userspace side most of the current activity on ffmpeg is focussed on making everything work. The process of usptreaming is looming on the horizon but it's still something to come. Again there's quite a bit of collaboration happening on this and we're involved.

Vulkan is an alternative to things like OpenGL/OpenGLES .. so "standards" .. https://xkcd.com/927/

Quote from chewitt

although you need to run under Wayland which doesn't support fun stuff like "adjust-refresh" by design so it's not quite the solution nVidia users need (better than "no support" though)..

I don't think wayland is a requirement for nvdec vaapi wrapper. However, I don't know if Kodi starts in GBM mode with new nvidia drivers.

offbeat In short, I don't think there will ever be one single go-to video decoding API under Linux, mostly becasue supported HW is very diverse. However, I believe it will boil down to few different apis.

Quote from chewitt

The x86_64 world is a little different to the ARM world. Intel/AMD standardised their hardware decoders around VAAPI so Kodi can call ffmpeg in a consistent way regardless of each, although at some point capabilities need to reflect which GPU/card and what the TV/panel supports. It all works reasonably well though advanced things like tonemapping still need to be plumbed in. Even nVidia finallly caved and added support for GBM/mesa to their latest drivers; and although Kodi does not support NVDEC (needed) someone wrote an NVDEC > VAAPI ship and we have a proof of concept implementation; although you need to run under Wayland which doesn't support fun stuff like "adjust-refresh" by design so it's not quite the solution nVidia users need (better than "no support" though)..

The ARM world is domainated by ARM Mali GPUs (well supported with FOSS drivers now) with separate vendor-specific decoders. These have a mix of "stateless" and "stateful" types under the V4L2 banner. The stateless V4L2 "requests" uAPI is stable and supports a good range of codecs upstream; it's still up to individual drivers to support them. The stateful "mem2mem" side is not declared stable yet, and with the dependency on vendor firmware blobs is generally a bit futher behind. There is a lot of collaboration happening on testing and upstreaming to the kernel - and LE is heavly involved in that (Kodi is a good test app).

On the userspace side most of the current activity on ffmpeg is focussed on making everything work. The process of usptreaming is looming on the horizon but it's still something to come. Again there's quite a bit of collaboration happening on this and we're involved.

Vulkan is an alternative to things like OpenGL/OpenGLES .. so "standards" .. https://xkcd.com/927/

Maybe a bit off-topic and maybe too philosophical, but do you see the vendor-specific ARM-based stuff as the 'future' of Kodi? I see distros like OSMC and (maybe to a lesser extent) ZDMC, et. all, supporting stuff more quickly simply because they pack the vendor-specific blobs in there and it 'mostly' works... I know HDR stuff in the Linux kernel/rendering pipeline has been a nightmare so far for generic stuff.

I ask mainly because I would love to build out another fanless x64 box for my home theater Kodi device (currently a Vero 4k+); I find the embedded systems to simply be under-powered.

Quote from isamudaison

do you see the vendor-specific ARM-based stuff as the 'future' of Kodi?

LibreELEC migrated from vendor specific codebase (RaspberryPi, Amlogic) to mainline. So this is past, not future.

Quote from isamudaison

I ask mainly because I would love to build out another fanless x64 box for my home theater Kodi device (currently a Vero 4k+); I find the embedded systems to simply be under-powered.

You shouldn't need 10W+ to play media and run a pretty skin either, efficiency is king here IMHO

Currently I think an S905X3 SoC is tough to beat in terms of 'sweet spot' capabilities and efficiency.

Quote from powerarmour

You shouldn't need 10W+ to play media and run a pretty skin either, efficiency is king here IMHO

Currently I think an S905X3 SoC is tough to beat in terms of 'sweet spot' capabilities and efficiency.

Simply not my experience when you throw in LiveTV, and a media library of multiple thousands of objects...

Quote from isamudaison

do you see the vendor-specific ARM-based stuff as the 'future' of Kodi?

Team Kodi has been pursuing a "standards" based approach since 2016 and it's the reason Android now works very well. Linux is iinherently more complicated but a lot of the kernel frameworks are now in place and stable. Supporting older and current hardware has challenges as it means rework to existing codebases and commercial interests rarely fund work on old chips. However Google forcing Android device vendors to get ever-closer to upstream means future chip generations will increasingly have support written around the stable APIs and convergence will come. ARM devices are increasingly performant, and Intel stuff is still available (at a price) for anyone needing more.

Quote from offbeat

Both devices got hardware beefy enough to run something like LibreElec (3D acceleration, hardware video decoding, enough RAM), yet both are rotting away, feature and security wise, waiting to become e-waste.

Who wants such 'future'?

Of course some old devices can have a long life, but you've got to have a developer who has the time to put in hard work and have it not be an utterly thankless task.

A lot of old drivers won't even build these days, every dog has it's day unfortunately...

Numerous Linux/X11 Display Drivers Can No Longer Even Properly Build - Phoronix

"Team Kodi" is more accurately described as "small group of developers doing their own thing with occasionally overlapping interests" and as is common with non-commercial FOSS projects there is not much product management and no schedule. One of the few things its developers have ever consistently agreed upon is the need to reduce the complexity and improve maintainability of the codebase. That's the prime driver and motivation for the V4L2 long-term technical direction. Linux is inherently a fragmented ecosystem so we need to follow standards and be an advocate and agent-of-change for the establishment of those standards.