A Cooperative Multiagent Reinforcement Learning Framework for Droplet Routing in Digital Microfluidic Biochips

Digital microfluidic biochips (DMFBs) have shown great advantages in automatically executing biochemical protocols through manipulating discrete nano/picoliter droplets which are transported in parallel to achieve high-throughput outcomes. However, because of electrode degradations, the droplet transportation may fail, causing incorrect fluidic operations. To perform safety-critical bio-protocols, the reliability of droplet transportation becomes an utmost concern for DMFBs. It has been shown by the previous works that a reliable transportation policy can be learned using reinforcement learning (RL)-based methods by capturing the underlying health conditions of electrodes and making online decisions. However, previous RL methods may fail to accomplish routing tasks with multiple droplets, because there is a lack of cooperation among different agents (each agent represents one droplet). To deal with this problem and scale RL methods to many droplets, this article proposes a new cooperative centralized learning and distributed execution multiagent RL (MARL) framework for droplet routing in DMFBs using value-decomposition networks (VDNs). Moreover, to speed up the training and decision process as well as apply our method in large biochips, we use a partial observation space where agents can only observe environment in a limited field of view (FOV) centered around themselves. Compared with the state-of-the-art approach, the superior performance of the proposed approach is demonstrated on different DMFBs in terms of success rate and average completion time. We also validate our method on large biochips (e.g., $\mathbf {50\times 50}$ DMFBs) with more droplets than state-of-the-art approach (e.g., ten droplets).