Switching magnetism using only electricity is of great significance for information applications but remains challenging. We find that, altermagnetism, as a newly discovered unconventional magnetism, may open an avenue along this effort. Specifically, to have deterministic switching, i.e., reversing current direction must reverse magnetic structure, parity symmetry has to be broken. We discover that due to their symmetry that depends on chemical environments, altermagnet devices may naturally carry the parity symmetry breaking required for deterministic electrical switching of magnetism. More importantly, we identify MnTe and FeS bilayers as candidate devices. This scheme will inspire further explorations on unconventional magnetism and potential applications.