Did some more testing with the same Rpi4: swaped a few SD Cards with different systems on them to check if the 5 Ghz Issue is related to LibreElec or a general problem (hardware defect for example):
1. LibreElec (10.0.4 & 11.0.0) -> no connection possible ("invalid-key")
2. Raspian 64 Bit (Feb 2023 build) -> connection works just fine
3. PINN (3.8.4) -> connection works just fine
During all 3 tests the pi was in the same location / did not move so it can't be the signal strength either.
I am not 100% sure but I think the problems started with LE 10.0.3 or 10.0.4 and persist through LE 11.0.0.