I hope this is not to late for you but i think this is a problem from the pi 4 itself. I found out it works like charm, if i run it on 1080p but if i use it with my tv screen (4K) it stops working pretty quick. The pi is still accessable via ssh but no image. So i set it to 1080p via SSH and rebooted - viola - works!