The position of the IR receiver can certainly have an impact, you can try rotating/moving your RPi - ideally the receiver should point directly to you.
Use of Hifiberry should be harmless, I'm using a GPIO receiver myself with the Cirrus logic audio card.
Since you mentioned streaming from NAS the linked issue immediatly came to my mind. Simply speaking: transferring data via USB can mess with GPIO IR receivers. And on RPi the (wired) ethernet chip is connected via USB.
While personally I haven't experienced that issue yet (but I mainly play from local USB HDD, sometimes stream from the internet) it's not unthinkable that you are hit by it.