Experimental comparison of a heterogeneous learning multi-agent system with a homogeneous one

This paper reports an experimental study on the behavior of a heterogeneous system consisting of learning agents. Although we intuitively consider a heterogeneous system is more robust than a homogeneous one, few experimental results have been reported on a learning multi-agent system. Thus we experimentally compare a heterogeneous system with a homogeneous one. The two hunter agents independently pursue a prey agent on a one-dimensional lattice, and learn to act optimally with Q-learning sharing their episodes. When the hunter agents have different learning rates, we consider the system is heterogeneous. We made experiments in environments with or without locally minimum, and found that the heterogeneous system performed as well as the best homogeneous one in the most cases.