Online Multi-Face Tracking With Multi-Modality Cascaded Matching

Tracking multiple faces online in unconstrained \nvideos is a challenging problem as faces may appear drastically \ndifferent over time and identities can be inferred only based \non information available from past frames. Previous tracking \nmethods focus on face information without reference to other \nmodality information such as a person’s overall body appearance, \nleading to suboptimal performance. In this paper, we propose a \nnew online multi-face tracking method, called online multi-face \ntracking with multi-modality cascaded matching (OMTMCM), \nto improve the tracking performance by using both face and body \ninformation. The proposed OMTMCM consists of two stages, \nnamely detection alignment and detection association. In the \nfirst stage, a detection alignment module is designed to align \nface detection with body detection from the same person for the \nsubsequent detection association. In the second stage, a cascaded \nmatching module is designed to associate face detections across \nframes to locate trajectory of each target face by using both face \nand body information. Specifically, aligned face-body detections \nin the current frame are matched in a cascade manner with body \nand face features that are selected from past frames and stored in \nthe designed feature memory. In this way, our method can track \nmultiple faces online with both face and body information while \neliminating the possibility of face detection and body detection \nfrom the same person being separately assigned with different \nidentities. Experimental results demonstrate our method is on par \nwith or better than other online tracking methods for multi-face \ntracking.