Deep Learning Enabled Video Semantic Transmission Against Multi-Dimensional Noise

As a novel communication paradigm to achieve efficient end-to-end communication over wireless channels, deep learning (DL) enabled semantic communication system has attracted increasing attention. However, semantic wireless communication systems are affected by multi-dimensional noise, including physical channel noise and semantic noise caused by unstable sources in the real environment. Unfortunately, the joint consideration of physical channel noise and visual semantic noise in video semantic transmission has not been studied yet in the current literature. In this paper, we propose a novel DL-enabled video semantic communication system called DVSC, which employs channel and spatial attention mechanisms to achieve self-adaptation for different channel conditions, mitigating the impacts of varying degrees of physical channel noise on system performance. Additionally, we introduce a feature refinement module based on an encoder-decoder architecture, and a semantic correction module based on a lightweight U-Net structure, which can address semantic ambiguity caused by visual semantic noise at the transceiver. We also design a loss function that considers both pixel and perceptual quality metrics to jointly optimize the DVSC in the pixel and semantic domains. Our extensive experimental results demonstrate that the DVSC can adapt to various channel conditions and outperform benchmark comparison schemes while spending less time on encoding and decoding. To reproduce our results, we make the source code publicly available at https://github.com/Coding-Cow/DVSC.git.