Multimodal Attention Framework for Context-Aware and Semantically Rich Image Captioning
作者
Nasser Gawfan,Waseem Ullah,Latif U. Khan,Mohsen Guizani
出处
期刊:Proceedings [Institute of Electrical and Electronics Engineers] 日期:2026-08-13卷期号:: 1-6
标识
DOI:10.1109/icip61757.2026.11630558
摘要
The exponential growth of visual data across digital systems poses many challenges for efficient data processing, scalable data storage, and semantic understanding in large-scale image and video data. Image captioning, positioned at the intersection of computer vision and natural language processing, offers a route to convert large-scale image repositories into meaningful textual understandings. Nonetheless, existing vision language models frequently face scalability issues and struggle with fine-grained object recognition and relational reasoning across diverse domains. To overcome these limitations, we develop an attention-driven image captioning framework that is uniquely designed for large scale data environments. The proposed framework integrates scene attention, object attention, and graph interaction attention modules built on BLIP embeddings and a selectively fine-tuned GPT-2 language model, enabling efficient contextual learning and robust cross-domain generalization. We experimentally evaluate the proposed framework on MSCOCO and NoCaps benchmarks demonstrate a relative improvement of 10.8% in CIDEr and a gain of 3.6% in SPICE over ClipCap, with a relative increase of 39.7% in CIDEr for out-of-domain generalization. Overall, the proposed framework shows a scalable solution for extracting structured, semantically rich information from large-scale multimodal datasets, advancing the integration of big data analytics and vision language modeling.