Modeling Cameras for Autonomous Vehicle and Robot Simulation: An Overview

Simulation is increasingly important in the development and testing of robots and autonomous vehicles as it opens the door for candidate navigation, perception, and sensor fusion algorithms to be expeditiously probed in complex and safety-critical scenarios. As most robots and autonomous vehicles make heavy use of cameras to perceive their surroundings, camera modeling becomes a prerequisite for the successful simulation of these autonomous agents. This contribution outlines the context in which camera models are used; provides a component-by-component analysis of the image acquisition pipeline along with algorithms used for its modeling; and closes with a discussion of data-driven approaches that embrace a different perspective on camera modeling.