BgSubNet: Robust Semisupervised Background Subtraction in Realistic Scenes

Background subtraction (BgS) is a problem for handling pixel-level identification of changing or moving entities in the field of view of a static camera system. Recent works have discovered superior generalization to unseen realistic scenarios by an approach called deep BgS, which employs deep neural networks (DNNs) on concatenations of image inputs and their backgrounds. However, due to a lack of large-scale foreground-background image datasets, the challenges of manually creating background masks for existing Internet-scale image datasets, and a general lack of attention to circumvent them, studies to exploit the generalization potentials of deep BgS have yet to be thoroughly conducted for unseen realistic scenes. Hence, our work proposes an analysis regarding the generalization capabilities of deep BgS, particularly by their binary constraints of pixel feature representations. Then, to address the lack of large-scale labeled data to induce generalization, we propose a strategy to generate a virtually limitless freeform dataset, follow up with the first self-supervised (SS) pretraining strategy for deep BgS, then extend it to a semisupervised framework called BgSubNet, which can support inference on unseen scenarios by self-supervision and mixed-domain training with real and synthetic data. Our results are experimentally validated at competitive results with CAMO-UOW and CDnet2014 datasets to show considerably high performance against existing frameworks.