Highly symmetric magnetic environments have been suggested to stabilize the magnetic information stored in magnetic adatoms on a surface. Utilized as memory devices such systems are subjected to electron tunneling and external magnetic fields. We analyze theoretically how such perturbations affect the switching probability of a single quantum spin for two characteristic symmetries encountered in recent experiments and suggest a third one that exhibits robust protection against surface-induced spin flips. Further we illuminate how the switching of an adatom spin exhibits characteristic behavior with respect to low energy excitations from which the symmetry of the system can be inferred.