Anchoring Morphological Representations Unlocks Latent Proprioception in Soft Robots

This research addresses the need for robust proprioceptive methods that capture the continuous deformations of soft robots without relying on multiple sensors that hinder compliance. The authors propose a vision‐based deformation learning strategy called latent proprioception, which anchors the robot's overall deformation state to a single internal reference frame tracked by a miniature onboard camera. Through a multi‐modal neural network trained on simulated and real data, the authors unify motion, force, and shape measurements into a shared representation in latent codes, inferring unseen states from readily measured signals. The experimental results show that this approach accurately reconstructs full‐body deformations and forces from minimal sensing data, enabling soft robots to adapt to complex object manipulation or safe human interaction tasks. The proposed framework exemplifies how a vision‐based deformable learning approach can inform and enhance robotics by reducing sensor complexity and preserving mechanical flexibility. The authors anticipate that such hybrid system codesign will advance robotic capabilities, deepen the understanding of natural movement, and potentially translate back into healthcare and wearable technologies for living beings. This work paves the way for soft robots endowed with greater autonomy and resilience. All codes are available at GitHub: https://github.com/ancorasir/ProSoRo.