Application of conditional generative adversarial networks toward time-efficient and high-fidelity imaging via multimode fibers

Several modes exist in multimode fibers due to their large core diameter. Due to this property, they can transport images from one point to another. However, due to mode and polarization mixing, they randomize any information at their input to form speckle patterns. Original images from these patterns can be successfully reconstructed using methods like phase conjugation, transmission matrix measurement, and deep learning. Deep learning techniques are attractive as they are less time-consuming and do not require complex phase measurements and setups. Recently, researchers have used attention blocks with U-NET architecture to regenerate images from speckles. In this case, the average structural similarity index measure (SSIM) of reconstructed images is 0.8772. However, the network is complex and has high computation costs. As conditional generative adversarial networks (CGAN) produce better results for image-to-image translation problems, scientists have used them to reconstruct images from speckles with an average SSIM of 0.8686. We have designed a CGAN model that is fast (1 hour training time, 9.4ms inference time), stable (no mode collapse), and produces high-accuracy results. We have created our own data sets by sending 60000 (MNIST and Fashion MNIST) images at fiber input using a spatial light modulator while simultaneously recording speckle patterns on camera. The average SSIM achieved in our case is 0.9010 for 5000 unseen MNIST test images, which is greater than the previously reported values. The high-fidelity and fast imaging using our CGAN model offers the potential for developing thin, minimally invasive endoscopes using multimode fibers.