An Empirical Study on Curriculum Learning for Reinforcement Learning-based Biomechanical Arm Control

Controlling high-dimensional musculoskeletal systems is challenging due to the large number of muscle actuators and the need for coordinated motor behavior. Recent work has shown that combining reinforcement learning with curriculum learning can improve performance on such tasks, yet the design of effective curricula remains an open question. In this work, we develop two curriculum strategies, Large-to-Small and Small-to-Large, against a no-curriculum baseline in order to control a musculoskeletal model of the arm during a task involving the tracing of geometric shapes. Both curriculum strategies significantly outperform the baseline, with success rates of 94% and 88%, in contrast to 42% without a curriculum. The best results are presented using the Large-to-Small strategy, which suggests that the agent learns better when it begins with larger geometrical shapes and gradually progresses on to smaller ones, where the tracing becomes more precise. Furthermore, we evaluate generalization to unseen instances, where the results demonstrate that curriculum-trained agents effectively generalize to novel shapes and scale variations, illustrating the robustness of the learned motor skills beyond the training conditions.The code for reproducing our experiments is accessible at https://github.com/AsmaeOUHSSAIN/CuRL-BioArm.git