计算机科学
人工智能
光流
时间分辨率
RGB颜色模型
核(代数)
计算机视觉
时差学习
帧(网络)
运动补偿
模式识别(心理学)
数学
电信
物理
量子力学
组合数学
强化学习
图像(数学)
作者
Yi Xiao,Qiangqiang Yuan,Kui Jiang,Xueying Jin,Jiang He,Liangpei Zhang,Chia‐Wen Lin
出处
期刊:IEEE Transactions on Circuits and Systems for Video Technology
[Institute of Electrical and Electronics Engineers]
日期:2024-04-01
卷期号:34 (4): 2789-2802
被引量:18
标识
DOI:10.1109/tcsvt.2023.3312321
摘要
Optical-flow-based and kernel-based approaches have been extensively explored for temporal compensation in satellite Video Super-Resolution (VSR). However, these techniques are less generalized in large-scale or complex scenarios, especially in satellite videos. In this paper, we propose to exploit the well-defined temporal difference for efficient and effective temporal compensation. To fully utilize the local and global temporal information within frames, we systematically modeled the short-term and long-term temporal discrepancies since we observe that these discrepancies offer distinct and mutually complementary properties. Specifically, we devise a Short-term Temporal Difference Module (S-TDM) to extract local motion representations from RGB difference maps between adjacent frames, which yields more clues for accurate texture representation. To explore the global dependency in the entire frame sequence, a Long-term Temporal Difference Module (L-TDM) is proposed, where the differences between forward and backward segments are incorporated and activated to guide the modulation of the temporal feature, leading to a holistic global compensation. Moreover, we further propose a Difference Compensation Unit (DCU) to enrich the interaction between the spatial distribution of the target frame and temporal compensated results, which helps maintain spatial consistency while refining the features to avoid misalignment. Rigorous objective and subjective evaluations conducted across five mainstream video satellites demonstrate that our method performs favorably against state-of-the-art approaches. Code will be available at https://github.com/XY-boy/LGTD.
科研通智能强力驱动
Strongly Powered by AbleSci AI