Deep reinforcement learning-based dynamic path planning of flexible needle in robotic puncturing

Flexible needle puncture path planning in surgical robots faces significant challenges, such as limited adaptability to multi-objective environments and poor real-time performance. These issues affect both the accuracy and efficiency of needle puncture, restricting its application in complex medical scenarios. This paper proposes a deep reinforcement learning-based method to improve flexible needle path planning. To address the limitations of traditional models in accurately capturing needle dynamics, a hierarchical tissue model based on the unicycle framework is designed, which integrates kinematic and mechanical models. This approach considers the varying forces from different tissues on the needle at various positions. A dynamic multi-objective environment and obstacle model are also constructed, along with a target prioritization scheme for multi-objective optimization. Additionally, a prioritized experience replay (PER) mechanism is introduced to improve data efficiency in the learning process. This method enhances the needle’s adaptability and robustness in dynamic environments. Simulation results demonstrate that the model improved real-time performance and precision in dynamic multi-objective environments, making intelligent decisions based on target priorities and accelerating the exploration of optimal strategies.