计算机科学
人工智能
计算机视觉
质量(理念)
图像质量
图像(数学)
全向天线
图像分割
特征提取
可视化
图像处理
多模式学习
质量评定
模式识别(心理学)
多通道交互
深度学习
语音识别
机器视觉
作者
Jiebin Yan,Jiayu Zhang,Jiale Rao,Lei Wu,Pengfei Chen,Yuming Fang
标识
DOI:10.1109/tmm.2026.3710109
摘要
Multimodal learning has garnered significant attention in the era of large models, particularly for their remarkable zero-shot generalization capabilities. This approach has also been extensively explored in the field of image quality assessment. However, its application in blind omnidirectional image quality assessment (BOIQA) remains relatively underexplored. Additionally, most existing BOIQA models rely on computationally intensive viewport or patch sequence generation, thereby limiting model flexibility and efficiency. To address these challenges, we propose an intuitive and effective BOIQA model with Optimal Prompt guided Multimodal Learning (OPML) that processes raw omnidirectional images in Equirectangular Projection (ERP) format without requiring any transformation. Specifically, OPML consists of an optimal prompt selection module for eliminating quality-irrelevant information, which mitigates the limitations of objective models when encoding lengthy text descriptions, and a hierarchical visual degradation modeling branch for capturing visual quality features, along with a multimodal quality capturing module for fusing multimodal features and predicting the global quality of an omnidirectional image. The proposed model demonstrates competitive performance on two large-scale OIQA databases, outperforming most state-of-the-art methods, and also shows great generalizability in group Maximum Differentiation (gMAD) competition methodology.
科研通智能强力驱动
Strongly Powered by AbleSci AI