Optimal utilization of integrated photovoltaic battery systems: An application in the residential sector

PhotoVoltaic (PV) panels have been increasingly favored by residential users in recent years, due to noticeable reductions in their costs. The PV systems become more effective when combined with battery packages, which store the energy produced by the PV systems for later use. This way, the PV systems are able to provide flexible and reliable services even when the peak demand for electricity misalign with the window of most efficient PV power generation. In this study, we develop an integrated charge/discharge scheme for lithium-ion batteries to maximize their total expected benefit. Specifically, we develop a Markov Decision Process (MDP) model to maximize the battery utilization, subject to uncertainty in weather conditions and electricity demands, while accounting for battery degradation due to calendar aging and charging/discharging cycles. Due to the extremely slow rate of degradation in batteries, the state space of the MDP is excessively large. To solve the problem efficiently, we establish structural properties of the MDP and exploit them to solve the problem. We improve the backward induction algorithm with established structural properties. We further improve the Deep Q-network (DQN) algorithm by proposing two novel algorithms, the augmented DQN (ADQN) algorithm and the stochastic augmented DQN (SADQN) algorithm. Computational results indicate that ADQN and SADQN solve the problem much faster than DQN, with better solution qualities. The ADQN and SADQN algorithms provide flexibility for practitioners in real-world implementations.