Transformer-based deep learning models for predicting permeability of porous media

The direct acquisition of the permeability of porous media by digital images helps to enhance our understanding of and facilitate research into the problem of subsurface flow. A complex pore space makes the numerical simulation methods used to calculate the permeability quite time-consuming. Deep learning models represented by three-dimensional convolutional neural networks (3D CNNs), as a promising approach to improving efficiency, have made significant advances concerning predicting the permeability of porous media. However, 3D CNNs require significant computational resources due to their extensive parameters, which limit studies to small-sized porous media, and their generalization capabilities are insufficiently explored. To address these challenges, we propose a novel CNN-Transformer hybrid neural network, merging a 2D CNN with a self-attention mechanism. Additionally, we incorporate physical information into digital images, constructing a PhyCNN-Transformer model to reflect the impact of physical properties on permeability prediction. In terms of dataset preparation, we employ the publicly available DeePore porous media dataset with sample size of 2563 cubic voxels and labeled permeability calculated by Pore network modelling (PNM). We compare the two transformer-based models with a 3D CNN in terms of parameter number, training efficiency, prediction performance, and generalization, and the results show significant improvement. By employing transfer learning, the well-trained transformer-based models proved capable of adapting to porous media with different sizes (achieving an R2 score of 0.9563 with 300 training samples), while the 3D CNN lacks this transferability.