Bayesian Sequential Optimal Experimental Design for Linear Regression with Reinforcement Learning

We perform a comparison study on Bayesian sequential optimal experimental design algorithms applied to linear regression in two unknowns. We transform the Bayesian sequential optimal experimental design problem into a reinforcement learning problem to determine the power of deep reinforcement learning algorithms against baselines including batch design, greedy design, dynamic programming, and approximate dynamic programming. Using KL-divergence to measure information gain in the unknown parameters, we construct objectives for each algorithm to maximize information gain. This work showcases novel comparisons between the aforementioned algorithms and provides a new application of reinforcement learning to Bayesian sequential optimal experimental design for inverse problems in linear regression with multiple parameters.