Integrating unsupervised learning and transformer for missing log prediction

ABSTRACT Well-logging data plays a crucial role in the exploration and extraction of subsurface resources. However, in practical applications, logging data often suffers from missing values or distortions due to geologic limitations. To achieve comprehensive subsurface modeling, it is essential to accurately reconstruct this missing data. We assume that logging responses from the same lithology exhibit similar patterns from a petrophysical perspective. Therefore, incorporating lithologic information into the logging attribute prediction tasks can enhance the prediction accuracy of the model. We design a geologically constrained transformer architecture where the self-attention mechanism enables the model to better understand the relationships between different depth points in the logging data, capturing the complex features of the subsurface structure more accurately. By encoding lithologic information as a prior geologic constraint and incorporating it along with the logging sequences into the transformer model, we achieve more accurate predictions for missing logging sequences. To address the challenge of missing lithologic data, we introduce the results of Toeplitz inverse covariance-based clustering (TICC) method as a substitute for actual lithologic data. The TICC results are used as a geologic constraint in the transformer model to guide the prediction process. Experiments demonstrate that the transformer combined with TICC technique achieves predictive performance comparable to using actual lithologic data, improving the accuracy of logging predictions. This approach provides an effective alternative for practical exploration where real lithologic data is not available. Furthermore, we enhance the predictive capability of the model by designing a regularized loss function that combines the mean-squared error with a Gaussian distribution constraint. Application results on field data confirm the reliability and practicality of the geologically constrained transformer model in accurately predicting acoustic logging.