Multimedia Fusion at Semantic Level in Vehicle Cooperactive Perception

The complete perception of the environment is an important prerequisite for realizing autonomous driving. However, due to the existence of obstructions and the limits of sensing range, perception ability of a single vehicle cannot meet the requirements of perception. In this paper, we attempt to expand the range of vehicle perception and eliminate blind areas by fusing multimedia based on vehicle network. Using a deep leaning method, we get a more in-depth understanding and extract critical information from vision system. We integrate multi-vehicle’s information into the ego-vehicle’s coordinate by merging unified bird-view maps, taking both graphic and semantic alignment as association. Furthermore, a transport protocol is designed to limit the information transmission amount. The experimental results on open data sets show that our approach can effectively fuse multimedia information in the multi-vehicle system, and improve the vehicle’s perception ability.