Short-Term Memory Active Learning for Drug Development

Abstract Active learning is a powerful approach for efficient data selection in machine learning, particularly valuable in domains such as drug discovery where data acquisition is costly. Traditional active learning methods continuously expand the training set with newly labeled data but do not revisit the utility of the previously added data points. Here, we introduce short-term memory active learning (SMAL), a novel active learning framework that can remove training data during learning. We conceptualized, implemented, and evaluated multiple forgetting strategies based on prediction error and uncertainty metrics derived from random forest out-of-bag estimates. We benchmarked SMAL across six absorption, distribution, metabolism, excretion, and toxicity (ADMET) drug development data sets, demonstrating improved or competitive performance compared to classical active learning. We found that SMAL creates more balanced training sets and can reduce labeling costs through its unique data recycling mechanism. Furthermore, when tested on data sets with artificially introduced label errors, SMAL shows robust performance and autonomously identifies and discards corrupted data points. Beyond ADMET tasks, SMAL also shows potential utility for virtual screening: across 99 virtual-screening data sets with simulated temporal splits from the SIMPD benchmark, SMAL matched or outperformed classical active learning in 97.7% of comparisons. These results establish forgetting as a valuable addition to active learning, enabling more robust and efficient model development─especially in noisy, resource-constrained real-world settings.