Synthesizing Realistic Snow Effects in Driving Images Using GANs and Real Data with Semantic Guidance*

Intelligent vehicle perception algorithms often have difficulty accurately analyzing and interpreting images in adverse weather conditions. Snow is a corner case that not only reduces visibility and contrast but also affects the stability of the road environment. While it is possible to train deep learning models on real-world driving datasets in snow weather, obtaining such data can be challenging. Synthesizing snow effects on existing driving datasets is a viable alternative. In this work, we propose a method based on Cycle Consistent Generative Adversarial Networks (CycleGANs) that utilizes additional semantic information to generate snow effects. We apply deep supervision by using intermediate outputs from the last two convolutional layers in the generator as multi-scale supervision signals for training. We collect a small set of driving image data captured under heavy snow as the translation source. We compare the generated images with those produced by various network architectures and evaluate the results qualitatively and quantitatively on the Cityscapes and EuroCity Persons datasets. Experiment results indicate that our model can synthesize realistic snow effects in driving images.