DiffSRL: Learning Dynamical State Representation for Deformable Object Manipulation With Differentiable Simulation

Dynamic state representation learning is essential for robot learning. Good latent space that can accurately describe dynamic transition and constraints can significantly accelerate reinforcement learning training as well as reduce motion planning complexity. However, deformable object have very complicated dynamics and is hard to be represented directly by a neural network without any prior physics information. We propose DiffSRL, an end-to-end dynamic state representation learning pipeline that uses differentiable physics engine to teach neural network how to represent high dimensional pointcloud data collected from deformable objects. Our specially designed loss function can guide neural network aware physics constraints and feasibility. We benchmark the performance of our methods as well as other state representation algorithms with multiple downstream tasks on PlasticineLab. Our model demonstrates superior performance most of the time on all tasks. We also demonstrate our model's performance in real hardware setting with two manipulation tasks on a UR-5 robot arm.