Optimal Sensor Data Fusion Architecture for Object Detection in Adverse\n Weather Conditions

A good and robust sensor data fusion in diverse weather conditions is a quite\nchallenging task. There are several fusion architectures in the literature,\ne.g. the sensor data can be fused right at the beginning (Early Fusion), or\nthey can be first processed separately and then concatenated later (Late\nFusion). In this work, different fusion architectures are compared and\nevaluated by means of object detection tasks, in which the goal is to recognize\nand localize predefined objects in a stream of data. Usually, state-of-the-art\nobject detectors based on neural networks are highly optimized for good weather\nconditions, since the well-known benchmarks only consist of sensor data\nrecorded in optimal weather conditions. Therefore, the performance of these\napproaches decreases enormously or even fails in adverse weather conditions. In\nthis work, different sensor fusion architectures are compared for good and\nadverse weather conditions for finding the optimal fusion architecture for\ndiverse weather situations. A new training strategy is also introduced such\nthat the performance of the object detector is greatly enhanced in adverse\nweather scenarios or if a sensor fails. Furthermore, the paper responds to the\nquestion if the detection accuracy can be increased further by providing the\nneural network with a-priori knowledge such as the spatial calibration of the\nsensors.\n