Suprathreshold visual psychophysics and structure-based visual masking

Current state-of-the-art algorithms that process visual information for end use by humans treat images and video as traditional "signals" and employ sophisticated signal processing strategies to achieve their excellent performance. These algorithms also incorporate characteristics of the human visual system (HVS), but typically in a relatively simplistic manner, and achievable performance is reaching an asymptote. However, large gains are still realizable with current techniques by aggressively incorporating HVS characteristics to a much greater extent than is presently done. Achieving these gains requires HVS characterizations which better model natural image perception ranging from sub-threshold perception (where distortions are not visible) to supra-threshold perception (where distortions are clearly visible). This paper reviews classical psychophysical HVS characterizations focused on the visual cortex (V1), pertaining to the contrast sensitivity function, summation, and masking, which have been obtained using unrealistic stimuli such as sinusoids and white noise. The direct applicability of these results to natural images is often not clear. Complementary results are then presented which have been obtained using realistic stimuli derived from or consisting of natural images, along with several applications of these results. Finally, a new structure-based masking model is proposed to model masking in homogeneous natural image patches as a function of the image type: textures, edges, or structures.