Materials property prediction with uncertainty quantification: A benchmark study

Uncertainty quantification (UQ) has increasing importance in the building of robust high-performance and generalizable materials property prediction models. It can also be used in active learning to train better models by focusing on gathering new training data from uncertain regions. There are several categories of UQ methods, each considering different types of uncertainty sources. Here, we conduct a comprehensive evaluation on the UQ methods for graph neural network-based materials property prediction and evaluate how they truly reflect the uncertainty that we want in error bound estimation or active learning. Our experimental results over four crystal materials datasets (including formation energy, adsorption energy, total energy, and bandgap properties) show that the popular ensemble methods for uncertainty estimation are NOT always the best choice for UQ in materials property prediction. For the convenience of the community, all the source code and datasets can be accessed freely at https://github.com/usccolumbia/materialsUQ.