Turns out I had to add "dtparam=audio=on" to the config.txt. Now I can open alsamixer but the settings aren't saved. I guess I have to mount something as rw to apply the settings? Haven't figured it out yet.
Edit: creating /var/lib/alsa/asound.state and using alsactl store 0 didn't work
Edit2: Also the audio channel mapping is wrong. "Rear" controls the Sub, "Side" controls Rear/Surround and "Woofer" + "Center" don't seem to do anything.