Explainable visual emotion recognition via modular reasoning

Affective computing still faces significant challenges in mapping complex visual features to emotional states with high interpretability. Although Multimodal Large Language Models (MLLMs) offer strong generalization, their high computational cost for fine-tuning constitutes a limitation. Moreover, MLLMs often function as “black boxes”, resulting in a lack of transparency in how they prioritize visual cues, especially in zero-shot settings. This work investigates these limitations by proposing a modular reasoning strategy to enhance interpretability and enable more disentangled affective reasoning under noisy cues and resource-constrained conditions. We introduce the Chain-of-Responsibility (CoR), a multi-agent framework that decomposes the affective inference process into specialized agents (Facial, Body, and Contextual). Unlike traditional end-to-end models or single-prompt reasoning, CoR employs modular decomposition designed to expose intermediate modality-specific analyses. A Synthesizer agent finally integrates these analyses using an explicit priority policy enforced through prompt design. This design enhances transparency and auditability of the decision process, allowing researchers to inspect how facial, bodily, and contextual cues contribute to the final prediction, while maintaining competitive performance in zero-shot settings.