OPeMer: One-Shot LLM Prompting and Execution-Driven Multimodal Explainable Reasoning

The Visual Question Answering (VQA) task requires not only accurate answers but also interpretable reasoning processes, particularly in real-world applications where transparency is critical. To reduce annotation and computational costs while maintaining interpretability, the Few-shot Multimodal Explainable VQA (FS-MEVQA) task has been introduced, which aims to generate explanations with limited supervision. In this work, we propose OPeMer (One-shot Prompting and Execution-driven Multimodal Explainable Reasoning), a code-based framework that leverages large language models (LLMs) to generate executable Python programs for multimodal reasoning in a oneshot setting. These programs interact with a lightweight Python API to process visual inputs, capture intermediate reasoning artifacts—such as object crops and spatial relations—and optionally call external tools for open-world visual understanding. The resulting execution traces are serialized and provided to the LLM via a secondary prompt, enabling the generation of coherent multimodal explanations grounded in both visual and textual evidence. Designed without reliance on handcrafted rules or large-scale supervision, OPeMer offers an efficient and extensible approach to explainable multimodal reasoning. Experimental results on the SME dataset demonstrate that OPeMer achieves strong answer accuracy and explanation quality, even when using cost-effective LLMs under limited supervision, suggesting its potential for scalable and interpretable VQA.