Intermittent loss of stereo

I've had a couple strange cases where the left side speakers stop working for one or two feeds but continue fine for the others.

Example, I'm driving along and listening to the Sirius XM radio and the left side stops working. I change to CD and it works fine. I change back to XM, no left side. I change back to CD and it is fine. After a while on CD, the left side stopped working. I changed back to XM and left side was fine. Changed back to CD and left side not working. At one point, I had no left side on XM and radio, only on CD. But most of time, it works fine with all three. It is intermittent issue that occurs about once in 2-3 weeks. I've now just had my 4th event.

I would think loose connection, except that it works fine on other feeds when it does act up.

Is it likely loose connection, or is it an indication the unit is going bad?