Okay so, I don't know what I did but it randomly started working. Things I did below in no particular order:
- Wipe TPM Embedded Security
- Disable EVERYTHING in Security > BIOS Sure Start
- Disable EVERYTHING in Security > Secure Boot Configuration
- Disable EVERYTHING in Security > Secure Platform Management (SPM)
- Disable "Physical Presence Interface," "System Management Command" & "Trusted Execution Technology (TXT)" in Security.
- Disable "Fast Boot" & "Network (PXE) Boot" in Advanced > Boot Options.
- Disable "HP Sure Recover" in Advanced > HP Sure Recover
- Disable "HP Application Driver" in Advanced
I also tried a different USB. I don't even know what could have caused it to work, but here we are. Also as far as I know, there is no clear way to enable legacy boot support on this desktop. It just sorta "works" at some point according to their support.
Any ways, good luck future users with this issue. I hope it works for you also. 
