DaFIR: Distortion-Aware Representation Learning for Fisheye Image Rectification

This paper focuses on fisheye image rectification. Existing learning-based solutions learn image representations that mix distortion features and content features. Since the distortion feature dominates the rectification process, we propose a novel distortion-aware representation learning framework, which decouples the distortion feature from the content feature, for fisheye image rectification. Specifically, we first pre-train a Vision Transformer with a supervised pre-text task, which regresses the distortion distribution map of a distorted image. The pre-training equips the Vision Transformer with the ability to capture distortion-related patterns. After that, the pre-trained model is fine-tuned to predict the pixel-wise flow map to rectify the fisheye images. Extensive experiments are conducted to evaluate our approach and verify our idea of feature decoupling. The experiment results demonstrate the state-of-the-art performance of our approach compared to existing algorithms, as well as its generality on real-world images. Our source code is publicly available at https://github.com/lzk9508/DaFIR.