SeisBERT: A pretrained seismic image representation model for seismic data interpretation

In this paper, we propose training a representation model (denoted as SeisBERT) over a large database of migrated seismic images and applying it to extract features to accelerate machine learning-based seismic interpretation downstream tasks in new seismic volumes. More specifically, SeisBERT has a vision transformer architecture, and its training is achieved as self-supervised learning by employing the technique of masked image modeling. Our model is not exactly the BERT for 1D sequence. It is a bidirectional transformer for 2D seismic data that treats each 2D patch of a seismic image as a component in a sequence. We demonstrate the versatility of SeisBERT in multiple downstream tasks including seismic image similarity, facies classification, salt-body detection, fault detection, and image conditioning in seismic volumes that are not included in the training database of SeisBERT. Improvements are observed in both prediction accuracy and generalization, compared to results from the baseline models trained on each specific task.