Thanks chewitt I was able to work out a solution after checking that thread. All I have to do is run
on boot and have passthough enabled in kodi and I'm able to use both the audio from my phone and kodi without them interfering with each other. Would you know if its possible to run the pactl command automatically when restarting? Possibly with autostart.sh (is that still a thing?).