Exploring a Distillation with Embedded Prompts for Object Detection in Adverse Environments

Efficient and robust object detection in adverse environments is crucial and challenging for autonomous agents. The current mainstream approach is to use image enhancement or restoration as a means of image preprocessing to reduce the domain shift between adverse and regular scenes. However, these image-level methods cannot guide the model to capture the spatial and semantic information of object instances, resulting in only marginal performance improvements. To overcome this limitation, we explore a Prompts Embedded Distillation framework, called PED. Specifically, a spatial location prompt module is proposed to guide the model to learn the easily missed target position information. Considering the correlation between object instances in the scene, a semantic mask prompt module is proposed to constrain the global attention between instances, making each aggregated instance feature more discriminative. Naturally, we propose a teacher model with embedded cues and finally transfer the knowledge to the original student model through focal distillation. Extensive experimental results demonstrate the effectiveness and flexibility of our approach.