Vision first evolved in the water, where the spectral content of light informs about viewing distance. However, whether and how aquatic visual systems exploit this "fact of physics" remains unknown. Here, we show that zebrafish use "color" information to suppress responses to the visual background. For this, zebrafish divide their intact ancestral cone complement into two opposing systems: PR1/4 ("red/UV cones") versus PR2/3 ("green/blue cones"). Of these, the achromatic PR1 and PR4, which are retained in mammals, are necessary and sufficient for vision. By contrast, the color-opponent PR2 and PR3, which are lost in mammals, are neither necessary nor sufficient for vision. Instead, they form an "auxiliary" system that spectrally suppresses the "core" drive from PR1 and PR4. Our insights challenge the long-held notion that vertebrate cone diversity primarily serves color vision and further hint at terrestrialization, not nocturnalization, as the leading driver for visual circuit reorganization in mammals.