Rain-component-aware capsule-GAN for single image de-raining

Images taken in the rain usually have poor visual quality, which may cause difficulties for vision-based analysis systems. The research aims to recover clean image content from a single rainy image by removing rain components without introducing any artifacts. Existing rain removal methods often model the rain component as noise, but it obviously has clear patterns instead of random noise. Motivated by this, we raise the idea to build modules to capture rain patterns for de-raining. A Rain-Component-Aware (RCA) network is proposed to capture the characteristics of the rain. We then integrate it into an image-conditioned generative adversarial network (image-cGAN) as a RCA loss to guide the generation of rainless images. This results in the proposed two-branch cGAN, where one branch aims at improving the image visual quality after de-raining, and the other aims at extracting rain patterns so that the rain could be effectively removed. To better capture the spatial relationship of different objects within an image, we incorporate the capsule structure in both generator and discriminator of cGAN, which further improves the quality of generated images. The proposed approach is hence named as RCA-cGAN. Benefited by the RCA loss based two-branch optimization and the capsule structure, RCA-cGAN achieves good de-raining effect. Extensive experimental results on several benchmark datasets show that the RCA network is effective to capture rain patterns and the proposed approach could produce much better de-raining images in terms of both subjective visual quality inspection and objective quantitative assessment.