Posts by offbeat

Rather global question, right now in the year of 2022, is there anything resembling general consensus on the unified Linux API for video hardware decoding and zero copy rendering?

I’m only talking about mainline kernel and major opensource players/frameworks/players.

Is it VAAPI only for big boy x64 AMD/Intel GPU’s? Browsers, players like mpv, kodi seem to use it by default.

Stateless V4L and DRM for various ARM GPU’s? I see Libreelec has a lot of patches for kernel, ffmpeg and kodi, it’s not really mainlined yet?

How does Vulkan video API fit in there?

Quote from tony013

My preferred way would have been to add a patch - but I don't know, how to do it.

Clone linux kernel git, add your changes to arch/arm64/boot/dts/rockchip/rk3399-rock-pi-4.dtsi, then make a patch using "git format-patch" or "git diff".

Drop your patch to LE directory in projects/Rockchip/patches/linux/ and rebuild LE.

Here is what I did to add my custom overlay in LE:

1) created device tree fragment, in my case adding missing H3 CPU frequency

/dts-v1/;
/plugin/;

/ {
    fragment@0 {
        target = <&cpu0_opp_table>;

        __overlay__ {
            compatible = "operating-points-v2";
            opp-shared;

            opp-1296000000 {
                opp-hz = /bits/ 64 <1296000000>;
                opp-microvolt = <1250000 1250000 1300000>;
                clock-latency-ns = <244144>; /* 8 32k periods */
            };
        };
    };
};

2) compiled it to binary

dtc -I dts -O dtb -o sun8i-h3-cpu-clock.dtbo sun8i-h3-cpu-clock.dts

3) remounted /flash in read/write mode and copied sun8i-h3-cpu-clock.dtbo to /flash/overlays/

4) added "FDTOVERLAYS /overlays/sun8i-h3-cpu-clock.dtbo" to extlinux.conf, just before APPEND line.

It only has to be done once, your custom overlay will be kept even after LE upgrades.

Quote from jernej

It stalls only once at the beginning but after that it plays normally.

In my tests on H6 and H3, every video stalls for a bit right after start of the playback. Very small freeze, typically for one or two frames, only noticeable in pan scenes. According to logs, Kodi does some video sync error correction.

It's solved by applying windowing/gbm: Add display clock by popcornmix · Pull Request #19601 · xbmc/xbmc · GitHub you suggested some time ago and enabling "Sync video to display" in settings. Hope these commits will make it to LE soon.

Quote from jernej

Sorry to say, but options that are not enabled by default are in second plan, there is enough work already for a (very) small team of developers interested in particular platform. Having said that, I always like to hear about such investigations.

Completely understandable, just trying to help with testing things and giving feedback. Really appreciate work LE/Kodi/ffmpeg/sunxi/mesa teams are doing to advance modern video stack on Linux.

Particular H6 issue definitely isn't a burning priority, since it can be worked around without impairing playback quality or any other LE functions, it only adds a bit of power consumption.

Btw, another (bad) workaround to H6 jerky output is disabling "Allow hardware acceleration with DRM PRIME".

Of course it's even worse than setting algorithmdirtyregions=0.

Auto refresh rate switching is one of the major selling points of LibreElec, could even be enabled by default IMHO.

Many of Android TV boxes can't do that, since there was no such API until recently.

It gives user ability to play 24/25/30Hz content on TV without forced convertion to 60Hz. As well as ability to use TV provided motion interpolation, some hate it, some love it, but its a big deal anyway.

Quote from jernej

offbeat your samples plays fine with my two monitors, one full hd and another 4k. It stalls only once at the beginning but after that it plays normally. Tested on PineH64 model B.

Hmm, did you have "Adjust display refresh rate = on" set?

Playing everything at default 60 FPS most displays have in EDID is fine, jerkyness only appers when Kodi switches output HZ to match content FPS.

Quote from jernej

Can you test if using performance governor helps? That's only conceivable difference between H3 and H6 that could make a difference.

Actually performance governor instead of schedutil makes it even more visible and easy to repeat, at least for 30FPS content on 30Hz.

I've made a short clip of how it looks on display, will send PM.

Quote from mike2002

jernej !

I have a problem with my Orange pi 3 and Pine H64. All my 1080P contents (movies and episodes) are jerky (not smooth). The only workaround I found is to put devices to 2160p and whitelist only 2160p resolution. That way all my contents play well.

Do you have this problem on your side ? This problem doesn't exist on my raspberry pi 4, orange pi PC and Orange pi PC2.

Yup, playing content at native 24/25/30 Hz is still broken on Kodi+H6, very jerky.

I even bothered to compile and test in mpv, using same drmprime path on H6, play silky smooth there.

Playback is also OK on H3 board, using both Kodi and mpv.

Its somehow related to the combination of Kodi stopping rendering GUI/OSD during playback and panfrost on H6.

Workaround is to force Kodi to render GUI even when hidden, set algorithmdirtyregions to 0 in advancedsettings.xml. Of course power consumption/temperature rises a bit.

Quote from heitbaum

Revert of the bad commit added for 2.4.105 package

Tested master branch, problem is fixed. Thank you!

Quote from mike2002

I am unable to play movies with it, I got a black screen on all movie I try to play

Confirming, latest H6 master goes black on video, GUI overlay and audio are OK.

Broken due to libdrm: update to 2.4.105 by heitbaum · Pull Request #5301 · LibreELEC/LibreELEC.tv · GitHub

Quote from jernej

However, maybe default frequency can be raised same way it is done for A64 (via DT)?

Thank you for suggestions!

I've upped Beelink GS1 GPU to 756MHz and 980mV.

950mV had a few panfrost freezes/errors, 980mV runs Kodi and glmark2 without any problems.

diff --git a/projects/Allwinner/patches/linux/0000-Beelink-GS1-GPU-756Mhz.patch b/projects/Allwinner/patches/linux/0000-Beelink-GS1-GPU-756Mhz.patch
new file mode 100644
index 0000000000..83895abb00
--- /dev/null
+++ b/projects/Allwinner/patches/linux/0000-Beelink-GS1-GPU-756Mhz.patch
@@ -0,0 +1,22 @@
+diff --git a/arch/arm64/boot/dts/allwinner/sun50i-h6-beelink-gs1.dts b/arch/arm64/boot/dts/allwinner/sun50i-h6-beelink-gs1.dts
+index b5808047d6e4..0978e186c8f7 100644
+--- a/arch/arm64/boot/dts/allwinner/sun50i-h6-beelink-gs1.dts
++++ b/arch/arm64/boot/dts/allwinner/sun50i-h6-beelink-gs1.dts
+@@ -105,6 +105,8 @@ &emac {
+
+ &gpu {
+       mali-supply = <&reg_dcdcc>;
++      assigned-clocks = <&ccu CLK_GPU>;
++      assigned-clock-rates = <756000000>;
+       status = "okay";
+ };
+
+@@ -246,7 +248,7 @@ reg_dcdca: dcdca {
+
+                       reg_dcdcc: dcdcc {
+                               regulator-enable-ramp-delay = <32000>;
+-                              regulator-min-microvolt = <810000>;
++                              regulator-min-microvolt = <980000>;
+                               regulator-max-microvolt = <1080000>;
+                               regulator-ramp-delay = <2500>;
+                               regulator-name = "vdd-gpu";

This board has 800MHz LPDDR3 module (AWL3A1632), at least my revision does.
So I've also upped RAM frequency, pll-ddr0 is now reported to be running at 1.584GHz

diff --git a/projects/Allwinner/patches/u-boot/0000-Beelink-GS1-DRAM-800MHz.patch b/projects/Allwinner/patches/u-boot/0000-Beelink-GS1-DRAM-800MHz.patch
new file mode 100644
index 0000000000..fdbea7d4cb
--- /dev/null
+++ b/projects/Allwinner/patches/u-boot/0000-Beelink-GS1-DRAM-800MHz.patch
@@ -0,0 +1,12 @@
+diff --git a/configs/beelink_gs1_defconfig b/configs/beelink_gs1_defconfig
+index 9f5265512b..794ea399c5 100644
+--- a/configs/beelink_gs1_defconfig
++++ b/configs/beelink_gs1_defconfig
+@@ -3,6 +3,7 @@ CONFIG_ARCH_SUNXI=y
+ CONFIG_SPL=y
+ CONFIG_MACH_SUN50I_H6=y
+ CONFIG_SUNXI_DRAM_H6_LPDDR3=y
++CONFIG_DRAM_CLK=800
+ CONFIG_MMC0_CD_PIN="PF6"
+ CONFIG_MMC_SUNXI_SLOT_EXTRA=2
+ # CONFIG_PSCI_RESET is not set

With these changes and performance governor Kodi GUI is now able to reach 60FPS, except in some graphics heavy screens.

Thermal wise, thats 50C to 60C while idle, this box has suprisingly adequate heatsink.
Power consumption difference between schedutil and performance is 0.3 Watts or so, insignificant.

Quote from jernej

Reason why OSD helps is probably because EGL provides vsync... I don't know. Maybe windowing/gbm: Add display clock by popcornmix · Pull Request #19601 · xbmc/xbmc · GitHub will help. IDK.

I've rebuild LE with patches from this PR and enabled "Sync playback to display", no visible changes at all.
23.976 FPS on 23.976 Hz is still stuttering, drawing something with EGL still fixes it.

Did some further tests on Beelink GS1: configured Kodi to always draw (algorithmdirtyregions=0), enabled GALLIUM_HUD, launched Kodi mainscreen and tried switching CPU governors.

Basicly lowering CPU clock frequency affects Kodi GUI rendering speed, quite visibly.
Panfrost performance is inadequate enough already, 30 FPS only, but default schedutil clocks down CPU (as it should, since load is low) and it pushes rendering speed even lower, to 20FPS. Frametime on schedutil is bouncy and erratic as well.
Loading even one CPU core to 100% with something like 'openssl speed' instantly improves Kodi GUI rendering 20->30FPS

Also, locking CPU frequency to max also gives like +20% performance to hardware video decoding.
P.S. Testing it was a pain, the way ffmpeg is packaged in LibreElec is sooo confusing, binary programs aren't included in base build at all and ffmpeg-tools addon uses completely different build options, without any hardware decoding patches...

# echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
# ffmpeg -hwaccel drm -hwaccel_output_format drm_prime -i 03-h264-1080-30p-100mbps.mp4 -f null -
frame=  452 fps= 89 q=-0.0 Lsize=N/A time=00:00:15.06 bitrate=N/A speed=2.96x


# echo schedutil > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
# ffmpeg -hwaccel drm -hwaccel_output_format drm_prime -i 03-h264-1080-30p-100mbps.mp4 -f null -
frame=  452 fps= 76 q=-0.0 Lsize=N/A time=00:00:15.06 bitrate=N/A speed=2.54x


# echo powersave > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
# ffmpeg -hwaccel drm -hwaccel_output_format drm_prime -i 03-h264-1080-30p-100mbps.mp4 -f null -
frame=  452 fps= 74 q=-0.0 Lsize=N/A time=00:00:15.06 bitrate=N/A speed=2.48x

Perhaps there are some issues with hardware clocks, lowering CPU frequency also downclocks some other parts of H6, no idea if it's memory controller or GPU/VE/GE whatever else.

IMHO for now the only sane workaround is to use performance governor instead of default schedutil, forcing CPU to max 1.8GHz.

Strange problem, playing 23.976 FPS video outputing to 23.976 Hz TV on Beelink GS1 is rather jerky, micro stutters twice a second or so.

Switching CPU governor from default schedutil to ondemand makes things even worse. Changing governor to fixed frequency like performance or powersave helps, most of stutters are gone, but not all. Tested on two different TVs, same result.

Making Kodi draw any OSD during playback completely solves this issue, no matter what governor is active.

Can send a short video sample with pan movement, if needed.

Any way to override/change vsync settings for DRM apps?

I've tried running kmscube/glmark2/kodi with vblank_mode=0, output is still capped at monitor 60fps, or fraction of it (30,15,12,10...) for slower renders.

A few more things about GS1 suspend.

Now that RTC is working, suspend/resume tests can be scripted using "rtcwake"... which isn't included in Libreelec, so had to to edit util-linux config, build and copy in binary.

for i in $(seq 100); do
echo "Suspend #$i"
./rtcwake -m mem -u -s 3
sleep 10
done

100 iterations of rtcwake worked just fine, panfrost refused to suspend every now and then, but system stayed alive.
Which got me thinking, why the hell suspending from Kodi is getting box to unwakeable state on panfrost failure?

It turns out, systemd suspend is trying to enter deep sleep first (echo mem > /sys/power/state), and if it fails, enters lighter software-only sleep mode (echo freeze > /sys/power/state). Can't really wake up from "freeze" on GS1: no physical button, no wake-on-lan, control isn't transferred to Crust. Not sure if kernel is still processing IR in "freeze" mode, in case it does, some key remap may help.

Anyway, for now solution is to force systemd to suspend using "mem" method only. OS won't go to unwakeable state then, even if GPU fails to suspend.

/storage/.config/sleep.conf.d/sleep.conf
[Sleep]
SuspendState=mem

jernej , perhaps it can be added to global /etc/systemd/sleep.conf for Crust supported boards?

Quote from jernej

Yes, devfreq for GPU is missing because all attempts to make it work failed.

Last message I've found on this issue, LKML: =?UTF-8?B?Q2zDqW1lbnQgUMOpcm9u?=: Re: devfreq and panfrost on Allwinner H6, says problem is finally fixed.
No followups with testing or mainlining these patches though.

Quote from jernej

BTW, Crust/Linux fixes were just merged for GS1.

Superb, thanks!

Another issue I see is that GS1 doesn't seem to be running GPU at full speed.
I've enabled debug OSD in Kodi, held remote right button to keep Kodi scrolling/redrawing, and tested in both stock Android (latest Kodi 19 APK) and Libreelec.

Android animations are fluid, easily capping at screens 60FPS rate. Libreelec animations are rather jerky, FPS counter only goes to ~20FPS.

Kernel log says GPU is running at 432Mhz, if I'm intepreting things right. Google sources say Mali T720 in H6 can be run at 600.. 650 up until 700 Mhz.
There doesn't seem to be any way to control GPU frequency/voltage dynamicaly, devfreq API is missing.

[    0.281057] vdd-gpu: supplied by vcc-5v
[    7.221561] panfrost 1800000.gpu: clock rate = 432000000
[    7.221586] panfrost 1800000.gpu: bus_clock rate = 200000000
[    7.314868] panfrost 1800000.gpu: mali-t720 id 0x720 major 0x1 minor 0x1 status 0x0
[    7.314880] panfrost 1800000.gpu: features: 00000000,10309e40, issues: 00000000,21054400
[    7.314885] panfrost 1800000.gpu: Features: L2:0x07110206 Shader:0x00000000 Tiler:0x00000809 Mem:0x1 MMU:0x00002821 AS:0xf JS:0x7
[    7.314889] panfrost 1800000.gpu: shader_present=0x3 l2_present=0x1
[    7.425222] [drm] Initialized panfrost 1.1.0 20180908 for 1800000.gpu on minor 1