PIMnet: A quality enhancement network for compressed videos with prior information modulation

In this paper, we propose a quality enhancement network for compressed videos, named as PIMnet, which can effectively use the spatio-temporal information of multiple frames to improve the video quality. The main idea of PIMnet is to use the Quantization Parameter (QP) and Delta Picture Order Count (ΔPOC) of multiple input frames to modulate the network, where QP can reflect the quality of frames and ΔPOC can reflect the temporal distance between neighboring frames and the current frame. In PIMnet, the modulated deformable convolution (DCNv2) is performed to align and fuse multiple input frames. The offsets of DCNv2 for alignment are obtained by the flow-guided offset prediction module and the masks of DCNv2 for fusion are obtained by the mask prediction module. The offset and mask prediction modules are modulated by prior information. Afterwards, the features obtained by DCNv2 are further used by the QE module to compute the enhanced result. Extensive experiments demonstrate that the proposed PIMnet can achieve superior performance in quality enhancement.