Flexible multimodal camera using a light field architecture

We present a modified conventional camera that is able to collect multimodal images in a single exposure. Utilizing a light field architecture in conjunction with multiple filters placed in the pupil plane of a main lens, we are able to digitally reconstruct synthetic images containing specific spectral, polarimetric, and other optically filtered data. The ease with which these filters can be exchanged and reconfigured provides a high degree of flexibility in the type of information that can be collected with each image. This paper explores the various tradeoffs involved in implementing a pinhole array in parallel with a pupil-plane filter array to measure multi-dimensional optical data from a scene. It also examines the design space of a pupil-plane filter array layout. Images are shown from different multimodal filter layouts, and techniques to maximize resolution and minimize error in the synthetic images are proposed.