Prompt-Image-Caption Consistency for AI-Generated Image Quality Assessment

With the advancement of generative AI, generative models such as Generative Adversarial Networks (GANs) and diffusion-based models have significantly enhanced the capabilities in generating photorealistic images. Despite significant progress in AI-Generated Images (AIGIs), the evaluation of AIGI quality remains underexplored in multiple aspects, including AI artifacts, unnatural content, and the availability of input text prompts. However, existing methods are insufficient to capture both feature representations and correlations between image and text for AIGI quality assessment in aspects of perceptual image quality and text-to-image alignment. To address this, we proposed a novel Prompt-Image-Caption Consistency (PICC) framework, designed to adapt a pre-trained vision-language model for AIGI quality assessment by considering both perceptual image quality and text-to-image alignment at the same time. This framework effectively explores the prompt-image-caption triplet by adapting image embedding and multimodal embedding via Quality-Aware Attention to capture quality-aware features and calculating consistency scores to build the correlations among the prompt, image, and caption. Additionally, we propose a multilevel strategy to integrate both local and global information from multiple prompt-image-caption triplets, further enhancing prediction performance. Extensive experiments on benchmark datasets, including AIGIQA-20K and AGIQA-3K, demonstrate that the proposed PICC achieves state-of-the-art performance compared to baseline methods. The code will be made publicly available.