Quantum Enhancements for AlphaZero

Reinforcement learning algorithms including AlphaZero are powerful artificial intelligence (AI) algorithms, but are known to be resource intensive and unable to train within a reasonable budget. Speeding up learning would be valuable to further expand application areas for real world problems. In this paper, we investigate two quantum computing methods to enhance AlphaZero, with the goal of speeding up training. We evaluate the results by playing the board game Othello, an adversarial multi-agent turn based game similar to the game Go. First, parameterized quantum circuits (PQC) have been shown to train hybrid quantum-classical reinforcement learning agents in standard benchmark environments. With this inspiration, we replace the classical neural network with a PQC quantum neural network (QNN) in the AlphaZero architecture. Second, tensor-network quantum circuits have been used to extract important features for convolutional neural networks (CNNs) in image classification tasks. Using this as inspiration, we use a tree tensor network (TTN) to extract features from the Othello game board, generating a new set of feature vectors for a classical neural network to estimate the policy and value. Results show both novel methods converge to master the game and achieve the same level of play compared to the classical AlphaZero agent.