Real-time multitarget visual tracking with an active camera

This paper presents a real-time surveillance system to track multiple moving objects by controlling a pan-tilt camera platform. In order to describe the relationship between the targets and camera in this surveillance system, the input/output hidden Markov model (HMM) is applied here in the well-defined spherical camera coordinate. Since the targets are hard to be distinguished with one single camera when they are close to each other, we extend the particle filter for multitarget tracking with depth level estimate to track interacting targets. The targets overlapping each other still can be tracked in the images captured by single camera. Furthermore, an optimal camera action selection strategy is proposed to track multitarget within its limited field of view. The maximization of mutual information for the action design is formalized and implemented by the Monte Carlo method. The overall performance has been validated in the experiments of real-time tracking.