Magnifi Max keep switching HDMI source

Just got a Polk Magnifi Max which is connected to my Sony TV via HDMI ARC. I then have my Xbox, cable and a media box connected to the Polk. My problem is that if I have more than 1 source that's turned on, the Polk will rotate through all these sources even if I'd just selected one source. The only way I can get it to stop switching is to only turn on only 1 source i.e. if I'm on the Xbox, I have to turn off the cable and media box. Is this suppose to be the way?