3D robotic navigation using a vision-based deep reinforcement learning model

Abstract In this paper, we address a problem of vision-based 3D robotic navigation using deep reinforcement learning for an Autonomous Underwater Vehicle (AUV). Our research offers conclusions from the experimental study based on one of the RoboSub 2018 competition tasks. However, it can be generalized to any navigation task consisting of movement from a starting point to the front of the next station. The presented reinforcement learning-based model predicts the robot’s steering settings using the data acquired from the robot’s sensors. Its Vision Module may be based on a built-in convolutional network or a pre-trained TinyYOLO network so that a comparison of various levels of features’ complexity is possible. To enable evaluation of the proposed solution, we prepared a test environment imitating the real conditions. It provides the ability to steer the agent simulating the AUV and calculate values of rewards, used for training the model by evaluating its decisions. We study the solution in terms of the reward function form, the model’s hyperparameters and the exploited camera images processing method, and provide an analysis of the correctness and speed of the model’s functioning. As a result, we obtain a valid model able to steer the robot from the starting point to the destination based on visual cues and inputs from other sensors.