I think you should try that. We have reports on the forum that different HDMI ports act differently. Usually the first HDMI port of the TV is the most capable one.
This led me to the solution. See above.
Only downside is that the AVR's OSD doesn't display on the TV, since it's connected to the unused (except for ARC) port on the TV.
So it would be slightly preferable to go back to the original setup, with he Pi connected through the AVR -- if I can get 4k60hz that way. I'll try it with the new HDMI 2.0-certified cables when they arrive.