Transformer-Based Models Aid Prediction of Transient Production of Oil Wells

_ This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 206537, “Development of Deep Transformer-Based Models for Long-Term Prediction of Transient Production of Oil Wells,” by Ildar R. Abdrakhmanov, Evgenii A. Kanin, SPE, and Sergei A. Boronin, Skolkovo Institute of Science and Technology, et al. The paper has not been peer reviewed. _ The authors of the complete paper propose a novel approach to data-driven modeling of transient production of oil wells, applying transformer-based neural networks trained on multivariate time series composed of various parameters of oil wells measured during exploitation. By tuning machine-learning models for a single well (ignoring the effect of neighboring wells) on open-source field data sets, the authors of the paper demonstrate that the transformer-based method outperforms recurrent neural networks (RNNs) with long- and short-term memory (LSTM) and gated recurrent-unit cells in the forecasting of bottomhole pressure dynamics. Introduction The authors apply a novel deep-learning algorithm called a transformer to build surrogate models for simulations of well performance. Transformer architecture initially was developed for natural-language processing problems. However, in recent years, researchers have adapted transformers for time-series forecasting. In contrast with RNNs, the transformer does not process data sequentially. Instead, it handles the entire sequence by a multihead self-attention mechanism. The transformer is much more computationally efficient than an RNN because its training can be performed using graphics processing units. Transformer networks allow for application of the transfer-learning technique. First, the model is trained to predict the target parameter (e.g., bottomhole pressure) using the production data from a certain well. Next, the tuned weights are used to initialize the training procedure on the data from a target well. In other words, the fine-tuning is performed based on the pretrained model. The described technique allows the model to transfer knowledge from one time-series forecasting problem to another, leading to acceleration of the training process and improvement of model-prediction capability. The recurrent networks are not well-suited for transferring learning because the hidden state of recurrent cells is not transferred commonly. Problem Formulation The authors consider a transient production of an oil well with arbitrary completion. In most cases, various parameters of the well are measured during its exploitation, including bottomhole and wellhead pressure and temperature, flow rates for each phase, choke size, and parameters of electric centrifugal pumps (if available). In this case, the authors propose an alternative approach to find the dependence between bottomhole pressure and flow rate that is based on deep-learning algorithms trained on available field data and that allows for obtaining quick predictions of bottomhole pressure or flow rates. The data-driven coupled model of a well and reservoir can be used to plan and optimize the production process. Also, the developed model can be used to estimate well and formation properties using a constant flow-rate response of a well.