So I have been picking away at this very issue and have some information. I recently upgraded VM Workstation from 15.5.0 to 17.5 and I had a LibreElex 9.2.6 OVA that was running and stopped after the VM Workstation upgrade. It starts and goes to a black screen. I moved the OVA over to another machine still running VM Workstation 15.5 and it started working again. I also tried installing the LibreELec 11.0.6 OVA on the VM Workstation 17.5 machine and it hung after starting. I moved that OVA over to the VM Workstation 15.5 machine and it worked fine.
So the issue is definitely related to Workstation 17.5 but there is also one other difference between the systems, which I believe may be the real issue. The Workstation 17.5 machine has an nVidia GeForce 710 card and the Workstation 15.5 machine has an ATI Radeon 5400 video card.
I need to do some testing with either upgrading the 15.5 to 17.5 with the ATI card, swap out the nVidia card to an ATI card or if you want me to move a working OVA back to Workstation 17.5 and grab some Kodi logs, just tell me what you need. With 2 systems and working OVAs I can easily do testing. I'd like to get this working on Workstation 17.5
Thanks,
Jeff