An investigation into high dynamic range imaging technologies

This thesis studies high dynamic range imaging (HDR) technologies. It covers techniques for creating HDR radiance map from photographs, tone mapping HDR radiance map for display and the evaluation of the quality of tone-mapped images. The influential technique introduced by Debevec and Malik has become the de facto standard for recovering high dynamic range radiance maps from photographs and has been widely used in research and commercial systems for over a decade. However, we have discovered an important defect in the original algorithm that will make this technique often fail to produce reasonable results in the extremely bright or dark regions of a scene. Therefore, we introduce a novel technique to correct this defect. Instead of the original algorithm where only pixel values from the photographs are used to guide the synthesis of the high dynamic range radiance map, we explicitly incorporate the shutter speed information of the camera. At each spatial pixel location, we estimate a “suitable shutter” that will make that location best exposed. A pixel’s contribution to the high dynamic range radiance value is not only a function of its value but also depends on the difference between shutter speed used to take the pixel and the estimated “suitable shutter” of that pixel. We also show that this new idea can be successfully used to directly fuse differently exposed photographs into a single low dynamic range image for display in conventional low dynamic range devices. Then, we present a novel tone mapping framework. In this framework, firstly we introduce a tone mapping fidelity principle which explicitly stipulates that tone-mapped image data should not only be visually enhanced but should also stay faithful to the original image. Second, this principle naturally translates tone mapping into a constrained optimization problem where a two-term cost function, one measures the difference between the tone-mapped image and a visually enhanced version of the image, and the other measures the difference between the tone-mapped image and the original image, is optimized. The relative weightings of the two terms in the cost function not only offers an insightful and simple mechanism to control the appearance of the tone-mapped image but also enables the introduction of spatially varying or uniform weighting functions thus unifying local and global tone mapping in a single framework. The HDR image is not directly viewable and dynamic range compression will unavoidably loose information. A saliency map analyses the visual importance of the regions and can therefore direct the tone mapping operators to preserve the visual conspicuity of the regions that should more likely attract visual attention. Therefore, we present a novel tone mapping method - Saliency Modulated High Dynamic Range Image Tone Mapping (SMTM). In SMTM, we have developed a very fast algorithm to first compute the visual saliency map of the high dynamic range radiance map and then directly use the saliency of the local regions to control the local tone mapping curve such that highly salient regions will have their details and contrast better protected so as to remain salient and attract visual attention in the tone-mapped display. We present experimental results to show that SMTM provides competitive performances to state of the art tone mapping techniques in rending visually pleasing low dynamic range displays. We also show that SMTM is better able to preserve the visual saliency of the HDR image and that SMTM renders high saliency regions to stand out to attract observers’ attention. Finally, to solve a difficult problem of evaluating tone mapping algorithms, we introduce a novel approach - pair comparison using Web 2.0 Technology. In this evaluation approach, we have developed a Web2.0 style system that enables Internet users from anywhere to evaluate tone-mapped HDR photos at any time. We adopt a simple paired comparison protocol, Internet users are presented a pair of tone-mapped images and are simply asked to select the one that they think is better or click a “no difference” button. These user inputs are collected in the web server and analysed by a rank aggregation algorithm which ranks the tone-mapped photos according to the votes they received. The advantages of this approach include the potential of collecting large user inputs under a variety of viewing environments rather than limited user participation under controlled laboratory environments thus enabling more robust and reliable quality assessment. We also present data analysis to correlate user generated qualitative indices with quantitative image statistics which may provide useful guidance for developing better tone mapping operators.