So I keep getting these issues and it's usually resolved by restarting my router which isn't helpful in getting to the bottom of it.
I've noticed this in the journalctl:
I can't see anything on my router logs which is weird. No other device is having this issue which is why I'm posting here. I suspect it's an issue between the two devices, I just need to figure out what it is.