Sooo, I am now running the Odroid N2+ with 2 GB and my H265/4k/HDR/Atmos videos are finally playing as smoothly as they should. This thing is a beast with 50% more power in the CPU Benchmark add-on, and I guess the whole hardware is just way more powerful.
As suggested by chewitt I also tried overclocking the Pi 4 according to https://forum.libreelec.tv since I am running it in their great passively cooled case:
arm_freq=1950
gpu_freq=650
over_voltage=5
But it did not help with the stuttering.
Since LibreELEC is only in alpha stadium for the Odroid and does not play those H265 videos at all for me, I switched to CoreELEC, which is identical and running perfectly so far.
So it could also be that CoreELEC has a "better" or just different (since it's a different SoC) implementation of decoding high-end H265 files?
But I guess I am in the wrong forum from now on.
Just wanted to let everyone know who are still looking for the "right box" to play videos like those mentioned.