End-to-End Human-Object Interaction Detection Based on Cascaded Transformer

Human-Object Interaction (HOI) detection is a sophisticated task in scene understanding, focused on identifying pairs of humans and objects while inferring the visual relationships between them. To address the performance limitations of current approaches that rely on a single encoder-decoder framework for simultaneous detection of humans, objects, and their interactions, this paper present a innovative end-to-end HOI detection method utilizing a cascaded transformer, which decomposes the HOI detection task into two subtasks, human-object detection and interaction classification. These achieved through a shared encoder along with two specialized cascaded decoder, one for detecting human-object pairs and another for classifying interactions. The human-object decoder employs two distinct queries to detect humans and objects. Besides, we design an attention-guided interaction embedding module to generate queries for interaction classification. By implementing task-specific query designs, we clearly delineate task boundaries and inter-task associative information, thus mitigating query information interference and enhancing detection precision. Extensive experiments and analysis show that our method surpasses previous methods, achieving remarkable results of 30.95 mAP on HICO-DET and 61.29 mAP on V-COCO, respectively.