A novel visual guidance framework for robotic welding based on binocular cooperation

• An initial welding position guidance framework based on uncalibrated visual servoing and binocular cooperation is studied. The proposed method combines the advantages of passive and active vision sensors, and improves the flexibility of the vision system. • The global camera first controls the robot to approach the workpiece through the proposed EKF based uncalibrated visual servoing. Then, the derived cooperative motion equation will lead the robot towards the IWP. • 2 The features extraction algorithms for realizing UVS are studied. The method to estimate the desired position of welding torch is discussed. And a compression strategy for Yolo-v4 is designed to realize real-time welding torch detection, which can be an optional solution for target detection tasks on computational-resource-limited devices. • The simulation experiments demonstrate the feasibility and stability of the proposed guidance framework. The comparison experiments indicate that the framework can increase the flexibility of the vision system while maintaining satisfactory accuracy. Visual sensor based welding guidance technology has played an essential part in intelligentized robotic welding. However, scanning trajectory dependence of the laser vision sensor (LVS) and low accuracy of the passive vision sensor limit the application of single visual sensor based guidance methods in practical welding guidance. Aiming at this problem, a global camera is introduced to LVS system, and a novel initial welding point (IWP) guidance framework based on binocular cooperation is put forward. The robot is first controlled by the global camera to approach the workpiece through the proposed extended Kalman filter based uncalibrated visual servoing (UVS). Then, the derived cooperative equation of the LVS and the global camera will lead the robot toward the IWP. Subsequently, the feature extraction algorithms for UVS are designed. A compression strategy for Yolo-v4 is present to realize real-time welding torch detection in small computing platforms, and the estimation method of desired features is analyzed. Finally, the feasibility of the proposed algorithm is verified through experiments. The proposed framework combines the advantages of rich information for global camera and high precision for LVS, and could be a feasible solution to realize fully autonomous welding guidance.