It would seem to me that someone with the skills and motivation could develop an audio/music only based skin with no unwanted video related GUI items. The Chorus web interface is certainly a nice one for music, something along those lines for remote control might be the right compliment. All this would be more of a KODI generic effort and not specific to LE.
I don't see any need to strip out any other LE components, the only downside is maybe more frequent updates than you'd desire.
Library management might be the only issue. Do you really want your music clients managing the library? Or do you want it server managed?