PROBAST+AI: an interpretation of the tool for assessing the quality, risk of bias and applicability of prediction models based on traditional or artificial intelligence methods

In recent years, artificial intelligence (AI) or machine learning (ML) methods have been increasingly used in the development and evaluation of clinical prediction models. Their algorithms differ from traditional regression modeling methods, resulting in significant limitations of the existing Prediction Model Risk of Bias Assessment Tool (PROBAST) for their evaluation. To address these limitations, the tool is updated to PROBAST+AI in 2025. PROBAST+AI extends the original framework to specifically address methodological challenges unique to developing or evaluating clinical prediction models based on AI/ML algorithms. PROBAST+AI assesses the quality of model development and the risk of bias in model evaluation across 4 domains: participants and data sources, predictors, outcome, and analysis, encompassing 16 and 18 signaling questions, respectively. Furthermore, the tool evaluates the applicability of the model across 3 domains: participants and data sources, predictors, and outcome. This article aims to compare the changes between the original and updated versions of the PROBAST tool, interpret the key content and items of PROBAST+AI, and apply the updated tool to evaluate an example clinical prediction model publication, to help domestic systematic review authors, clinicians, and policymakers critically appraise studies that develop or evaluate prediction models based on traditional or AI/ML methods.