Autonomous, Vision‐based Flight and Live Dense 3D Mapping with a Quadrotor Micro Aerial Vehicle

The use of mobile robots in search‐and‐rescue and disaster‐response missions has increased significantly in recent years. However, they are still remotely controlled by expert professionals on an actuator set‐point level, and they would benefit, therefore, from any bit of autonomy added. This would allow them to execute high‐level commands, such as “ execute this trajectory ” or “ map this area .” In this paper, we describe a vision‐based quadrotor micro aerial vehicle that can autonomously execute a given trajectory and provide a live, dense three‐dimensional (3D) map of an area. This map is presented to the operator while the quadrotor is mapping, so that there are no unnecessary delays in the mission. Our system does not rely on any external positioning system (e.g., GPS or motion capture systems) as sensing, computation, and control are performed fully onboard a smartphone processor. Since we use standard, off‐the‐shelf components from the hobbyist and smartphone markets, the total cost of our system is very low. Due to its low weight (below 450 g), it is also passively safe and can be deployed close to humans. We describe both the hardware and the software architecture of our system. We detail our visual odometry pipeline, the state estimation and control, and our live dense 3D mapping, with an overview of how all the modules work and how they have been integrated into the final system. We report the results of our experiments both indoors and outdoors. Our quadrotor was demonstrated over 100 times at multiple trade fairs, at public events, and to rescue professionals. We discuss the practical challenges and lessons learned. Code, datasets, and videos are publicly available to the robotics community.