It seems to happen to me on heroes with at least one empty gear loadout. (Unfortunately, I never noticed if the spell part of the loadout was also empty.) This could just be a coincidence. But it is the only “event” that I have personally noticed.
Another thing I’ve noticed (again, coincidence?) is that if I have an empty loadout, and it switches places, it will generally switch back in a different session. If I fill the empty loadout before it switches back, it doesn’t seem to do the swap back.