MTBU-Bench: a Multimodal Benchmark for Evaluating Video-LLMs in Teacher Behavior Understanding

Video-LLMs hold significant potential for analyzing classroom dynamics to inform pedagogy and drive educational innovation. There is an urgent need to assess the capability of Video-LLMs in educational applications to identify their potential and limitations. Existing educational evaluation datasets often suffer from three major shortcomings: reliance on small-scale, single-label human annotations; a focus on unimodal analysis; and the absence of granular annotations required for multi-step reasoning. To address these gaps, we first introduce an efficient Human-LLM collaborative pipeline for high-quality multimodal annotation. We then propose MTBU-Bench, a comprehensive multimodal benchmark designed to assess teacher behavior understanding. It comprises over 300 classroom scenarios across 15 K-12 and higher education disciplines, organized into four evaluation tasks: Action Recognition, Temporal Boundary Extraction, Spatial Localization, and Behavioral Description. Extensive evaluations of twelve mainstream Video-LLMs reveal critical shortcomings in fine-grained temporal understanding, specifically in the precise localization of action boundaries and the discrimination of similar behaviors. Our benchmark and findings provide a foundation for the targeted development and application of Video-LLMs in educational research. The code and dataset are publicly available at our project homepage: https://github.com/271213/MTBU-Bench.