We introduce a wideband light-emitting diode (LED) illuminated snapshot on-axis reflective digital holographic microscopy (DHM) system where the fringe contrast in holograms can be optimized to achieve the highest phase sensitivity through polarization manipulation. The DHM system is constructed based on a compact Michelson interferometer, utilizing a long working distance objective. A snapshot of four phase-shifted on-axis holograms is achieved by combining polarization phase shifting interferometry and a commercial pixelated polarization camera. The extremely short coherence length of the LED enables quantitative phase imaging with an ultra-low spatial noise level. Measurements of various samples verified the validity and capacity of the presented method.