Rolling Shutter Camera: Modeling, Optimization and Learning

Most modern consumer-grade cameras are often equipped with a rolling shutter mechanism, which is becoming increasingly important in computer vision, robotics and autonomous driving applications. However, its temporal-dynamic imaging nature leads to the rolling shutter effect that manifests as geometric distortion. Over the years, researchers have made significant progress in developing tractable rolling shutter models, optimization methods, and learning approaches, aiming to remove geometry distortion and improve visual quality. In this survey, we review the recent advances in rolling shutter cameras from two aspects of motion modeling and deep learning. To the best of our knowledge, this is the first comprehensive survey of rolling shutter cameras. In the part of rolling shutter motion modeling and optimization, the principles of various rolling shutter motion models are elaborated and their typical applications are summarized. Then, the applications of deep learning in rolling shutter based image processing are presented. Finally, we conclude this survey with discussions on future research directions.