Autonomous object tracking with vision based control using a 2DOF robotic arm

Abstract The tracking of moving object by implementing robot manipulator is one of the challenging task for many applications such as manufacturing, agriculture, logistics, healthcare, space, military, entertainment, etc. In the deployment of robotic manipulators with real-time object tracking for aforementioned important applications, the proper sensor surveillance and ensuring stability are major challenges. The purpose of this study is to design a precise and responsive object-tracking system by eliminating the complexities related to tedious mechanisms, rigidity, requirement of multiple sensors, etc. which are commonly associated with traditional systems. The robotic arms can be effectively designed to track moving objects autonomously with vision-based control. In comparison with different classical and traditional servoing approaches, the image-based visual servoing (IBVS) is more advantageous in vision-based control. The present article describes a new approach for IBVS-based tracking control of 2-degree-of-freedom (DOF) robotic arm by including object identification and trajectory tracking based crucial components. To solve the issues associated with IBVS, an accurate deep learning-based object detection framework is employed. The presented framework is utilized to detect and locate the objects in real-time. Further, an effective vision-based control technique is designed to control the 2-DOF robotic arm with the help of real-time response of object detection system. The validation of proposed control strategy is done by performing a simulation and experimental investigations with CoppeliaSim robot simulator and 2-DOF robotic arm, respectively. The findings reveal that the proposed deep learning controller for the vision-based 2-DOF robotic arm achieves good levels of accuracy and response time while performing visual servoing tasks. Furthermore, thorough discussion on possibility of using data-driven learning technique has been explored to improve the robustness and adaptability of the presented control scheme.