Action-Perturbation Backdoor Attacks on Partially Observable Multiagent Systems

Backdoor attacks on reinforcement learning implant a backdoor in a victim's policy. Once the victim observes trigger signals, it switches to the abnormal mode and fails its task. Most existing attacks assume the adversary can arbitrarily modify the victim's observations (i.e., vision-perturbation), which may be impractical. A more practical way is letting one adversary agent use its actions to affect the victim's observations (i.e., action-perturbation). However, in partially observable multiagent systems, agents may not always be able to observe others. When and how much the adversary agent can affect others' observations are uncertain. Also, we want the adversary agent to trigger others' backdoors with a few actions for stealthiness. To achieve that, we first design a novel training framework to produce auxiliary rewards that measure the influence of each trigger action on others' observations. We then use these rewards to train a trigger policy that guides the adversary agent to efficiently affect others' observations. Given the affected observations, we train the other agents to perform abnormally. Experiments show that the proposed method enables the adversary agent to trigger others' backdoors efficiently. Finally, we study the defense methods against the attack and provide insights for the deployment of multiagent systems.