RSPMP: real-time semantic perception and motion planning for autonomous navigation of unmanned ground vehicle in off-road environments

Considering autonomous navigation of an unmanned ground vehicle (UGV) in off-road environments, it faces various problems, such as semantic perception and motion planning. This paper proposes an intelligent approach to perception and planning for UGV in field environments. Firstly, a semantic image of environment is generated in real time based on an improved Convolutional Neural Network (CNN). Secondly, we provide two practical extensions to an open-source 3D mapping framework. One is the semantic point cloud fusion based on 3D LIDAR and Camera, and the other is the generation of traversability cost map using both semantic and geometric information. Thirdly, we propose a new kinodynamic semantic-aware planner which adds the dynamic window approach to the receding horizon planner so that the latter can meet the kinodynamic while perceiving semantic labels. Finally, the above methods, along with a localization module, are integrated into a complete autonomous navigation system with real-time semantic perception and motion planning (RSPMP). In the experiments, the proposed method was successfully applied for safe autonomous perception navigation in off-road environments.