Multi-focus image fusion: A Survey of the state of the art

Abstract Multi-focus image fusion is an effective technique to extend the depth-of-field of optical lenses by creating an all-in-focus image from a set of partially focused images of the same scene. In the last few years, great progress has been achieved in this field along with the rapid development of image representation theories and approaches such as multi-scale geometric analysis, sparse representation, deep learning, etc. This survey paper first presents a comprehensive overview of existing multi-focus image fusion methods. To keep up with the latest development in this field, a new taxonomy is introduced to classify existing methods into four main categories: transform domain methods, spatial domain methods, methods combining transform domain and spatial domain, and deep learning methods. For each category, representative fusion methods are introduced and summarized. Then, a comparative study for 18 representative fusion methods is conducted based on 30 pairs of commonly-used multi-focus images and 8 popular objective fusion metrics. All the relevant resources including source images, objective metrics and fusion results are released online, aiming to provide a benchmark for the future study of multi-focus image fusion. Finally, several major challenges remained in the current research of this field are discussed and some future prospects are put forward.