A really weird bug : If a disable AC3 transcoding, I set the number of channel to 7.1 and I watch an episode or a movie in AAC 5.1, I hear only the background music, no dialog... But if I set number of channel to 2.0 and I enable AC3 transcoding everything is fine.
After recent investigation, I found out that this makes perfect sense. ALSA config is written in such way, that all audio is transcoded in IE958 format. This is a bit problematic - DW-HDMI IEC958 support is meant only for compressed formats, 2 ch for normal compressed streams, 8 ch for HBR. It's unclear at this point if any other channel combination may work with IEC958 format. With AC3 transcoding you make sure that there is no uncompressed audio, so IEC958 should work fine in all cases. GUI uses stereo sound, which should work with IEC958 too.