Deep learning modeling in microscopy imaging: A review of materials science applications

The accurate analysis of microscopy images representing various materials obtained in scanning probe microscopy, scanning tunneling microscopy, and transmission electron microscopy, is in general time consuming as it requires the inspection of multiple data bases for the correct interpretation of the observed crystal structures. This task is especially demanding in microscopy video analysis involving a vast amount of image data. The recent development of deep learning (DL) algorithms has paved the way for cutting-edge microscopy studies in materials science, often outperforming conventional image analysis methods. This paper reviews the state-of-the-art in DL-based synthetic data generation, materials structure identification, three-dimensional structural reconstruction, and physical properties evaluation for different types of microscopy images. First, the fundamental concepts of DL relevant to materials science applications are reviewed. Subsequently, the combined experimental measurements and numerical simulations for preparing dedicated microscopy image for DL analysis are discussed. Then, the review concentrates on the core topic of the paper, that is the critical assessment of DL advances in materials' structural and physical properties evaluation. We believe that the future development and deployment of DL for practical microscopy data analysis will rely on the progress and improvement of advanced algorithms and innovative methods for training data generation.