PCD reconstruction, object classification and pose estimation for underwater vehicles using orthogonal multibeam forward looking sonar fusion

Orthogonal Multibeam Sonar Fusion (OMSF) is a recent fusion based method capable of producing accurate underwater 3D Point Cloud Data (PCD) from Multibeam Forward Looking Sonars (MFLS), enabling accurate seabed mapping and object scanning. This article provides methodical testing of OMSF reconstruction for MFLS accounting for operating frequency effects and object shapes. The article then proposes novel perception based applications for OMSF, consisting of classification technique integrating OMSF PCDs with a PCD based Convolutional Neural Network (CNN), and pose estimation method combining Orthogonal Feature Matching (OFM) bounding box regression with a pose regression CNN. Reconstruction test results show that OMSF is more accurate and efficient using higher frequency MFLS and achieves up to 53% higher accuracy on solid surfaces compared to hollow frames. Application tests based on an underwater garage dock show classification using OMSF PCDs can achieve 25% and 37% higher success rate and confidence while being more efficient, compared to using raw 3D sonar data. OFM bounding box regression produces 4.28% higher mean Intersection over Union (IoU), and 10% increase in ¿25% IoU metric compared to methods based on more traditional MFLS filtering. Similarly, end-to-end pose estimation achieves 6.25% higher success rate with OFM bounding box samples compared to ones obtained using traditional MFLS filtering.