Generative Test-time Augmentation Framework with Multi-Agent Collaboration for Zero-Shot Multimodal Named Entity Recognition

Multimodal Named Entity Recognition (MNER) aims to identify entities and their types from text–image pairs, using images to resolve ambiguity in short social media posts. Existing approaches rely heavily on large-scale annotated datasets and often struggle to generalize to unseen entities. To address these limitations, we formalize the zero-shot MNER (ZeroMNER) task, where models must perform entity recognition without any labeled data. We adapt large language models (LLMs) to this setting by leveraging their strong generalization capabilities. However, our experiments show that LLMs remain inadequate for ZeroMNER, despite performing substantially better in few-shot scenarios. Motivated by this gap, we propose GTA-MA, a Generative Test-time Augmentation Multi-Agent framework that reformulates ZeroMNER as a pseudo in-context learning (ICL) problem. GTA-MA synthesizes test-aligned multimodal demonstrations at inference time, conditioned on each input instance, ensuring strong semantic relevance to the target sample and improving ICL effectiveness. GTA-MA further employs coordinated agents to generate, annotate, select, and exploit these demonstrations, forming reliable pseudo in-context supervision that enables LLMs to self-hint without requiring labeled data. Experiments on Twitter-2015 and Twitter-2017 demonstrate substantial improvements, with F1 gains of 15.13 and 14.43 points over vanilla zero-shot baselines, respectively. Moreover, GTA-MA achieves performance comparable to few-shot approaches while consistently outperforming existing zero-shot methods.