MWFormer: Multi-Weather Image Restoration Using Degradation-Aware Transformers

图像复原 计算机科学 图像处理 降级(电信) 计算机视觉 人工智能 变压器 图像(数学) 工程类 电信 电气工程 电压
作者
Ruoxi Zhu,Zhengzhong Tu,Jiaming Liu,Alan Conrad Bovik,Yibo Fan
出处
期刊:IEEE transactions on image processing [Institute of Electrical and Electronics Engineers]
卷期号:33: 6790-6805 被引量:85
标识
DOI:10.1109/tip.2024.3501855
摘要

Restoring images captured under adverse weather conditions is a fundamental task for many computer vision applications. However, most existing weather restoration approaches are only capable of handling a specific type of degradation, which is often insufficient in real-world scenarios, such as rainy-snowy or rainy-hazy weather. Towards being able to address these situations, we propose a multi-weather Transformer, or MWFormer for short, which is a holistic vision Transformer that aims to solve multiple weather-induced degradations using a single, unified architecture. MWFormer uses hyper-networks and feature-wise linear modulation blocks to restore images degraded by various weather types using the same set of learned parameters. We first employ contrastive learning to train an auxiliary network that extracts content-independent, distortion-aware feature embeddings that efficiently represent predicted weather types, of which more than one may occur. Guided by these weather-informed predictions, the image restoration Transformer adaptively modulates its parameters to conduct both local and global feature processing, in response to multiple possible weather. Moreover, MWFormer allows for a novel way of tuning, during application, to either a single type of weather restoration or to hybrid weather restoration without any retraining, offering greater controllability than existing methods. Our experimental results on multi-weather restoration benchmarks show that MWFormer achieves significant performance improvements compared to existing state-of-the-art methods, without requiring much computational cost. Moreover, we demonstrate that our methodology of using hyper-networks can be integrated into various network architectures to further boost their performance. The code is available at: https://github.com/taco-group/MWFormer.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
刚刚
canhui发布了新的文献求助10
1秒前
canhui发布了新的文献求助10
1秒前
canhui发布了新的文献求助10
1秒前
souven发布了新的文献求助10
1秒前
canhui发布了新的文献求助10
1秒前
canhui发布了新的文献求助10
1秒前
canhui发布了新的文献求助10
1秒前
2秒前
北23333完成签到,获得积分10
2秒前
2秒前
感动霸完成签到 ,获得积分10
2秒前
3秒前
3秒前
wsb76完成签到 ,获得积分10
3秒前
phd_cheng发布了新的文献求助10
3秒前
4秒前
2024020847发布了新的文献求助10
4秒前
4秒前
虎浪嘿发布了新的文献求助10
4秒前
传奇3的应助被贪恋朝夕y采纳,获得10
4秒前
4秒前
5秒前
elvira给舒适的秋尽的求助进行了留言
6秒前
闪闪的梦槐完成签到,获得积分10
6秒前
7秒前
天天发布了新的文献求助20
7秒前
shidewu发布了新的文献求助10
8秒前
8秒前
8秒前
jhcdgszjdcb发布了新的文献求助10
9秒前
Aaiz发布了新的文献求助10
10秒前
干干完成签到,获得积分10
10秒前
souven完成签到,获得积分10
10秒前
不能说的秘密完成签到,获得积分10
10秒前
11秒前
11秒前
科研通AI6.3的应助被Harvest采纳,获得10
12秒前
1733发布了新的文献求助10
12秒前
12秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
Grapes 1000
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
John McDowells Bildungstheorie 800
八旗制度の研究 600
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8178484
求助须知:如何正确求助?哪些是违规求助? 9569318
关于积分的说明 21803439
捐赠科研通 7773381
什么是DOI,文献DOI怎么找? 3432819
关于科研通互助平台的介绍 2556838
邀请新用户注册赠送积分活动 2478441