Person Detection Based Adaptive Video Synthesis: A Fusion Framework for Ultra HD Panorama and Green Screen Videos

The past two years have witnessed the increasing prevalence of metaverse, while the COVID-19 pandemic has accelerated the formation of a non-contact culture. Under this circumstance, virtual reality once again attracts the public attention. Panorama video, as one of the most important forms of virtual reality, provides users with excellent immersion experience. This paper has proposed a fusion framework for Ultra HD panorama videos and green screen videos. In this framework, panorama videos are set as the virtual background layer on which the user-defined real foreground layer of portrait obtained by green screen matting is superimposed. During video fusion process, the portrait size is adaptively determined by parameters provided by a person detection algorithm running on panorama videos. Therefore, a more natural video synthesis result can be achieved, and presented on a head-mounted display or a flat screen device to provide an indistinguishable visual experience to the users.