Using the words 'performance' and 'Pi3' in the same sentence is wrong to begin with. The RPi's are 35 dollar/euro kids toys, and should not be compared for performance to anything like an Intel PC.
The Rpi has only a 100Mb/s network connection, which due to its design cannot even be used to its full potential. Also, any decent HDD will be faster than the (theoretical) 12.5MB/s network speed on a 100Mb/s connection. The slowest part in your whole network will always be the weakest link. In this case, the RPi's network. But I guess flac files are relatively small compared to video files, so it's all doable.
If you are planning to use a RPi as a server with anything Linux, I would choose the Linux default EXT4 format. But I'm not totally objective, I've been using Linux for some 8 years now. If you need to attach the HDD to a PC, you can always run a Live Ubuntu Linux session from a bootable USB drive and have access to the EXT4 partition(s).