What Do Generative Models Learn About Adaptive Immune Receptor Repertoires? A Benchmark Study

Generative models are increasingly used to model adaptive immune receptor repertoire (AIRR) sequence distributions, promising to decode the sequence diversity shaping immune responses and accelerate the design of therapeutic antibodies and T-cell receptors. Yet it remains unclear whether these models produce biologically meaningful outputs or merely capture surface-level sequence statistics while missing features driven by receptor generation and selection. Rigorous evaluation is needed, but the field lacks established standards, as existing machine learning metrics do not all translate directly to the AIRR domain, given the complex structure of the data and the lack of biological ground truth. Consequently, researchers face difficulties in evaluating the models and selecting appropriate ones, which can critically affect downstream clinical applications. Here, we apply a suite of evaluation metrics tailored to AIRR sequence data and present a systematic comparison of popular generative model families proposed for the AIRR field, including variational autoencoders, long short-term memory networks, antibody language models, selection models, and simple statistical baselines. We focus specifically on the task of learning individual-specific immune receptor repertoires, a clinically relevant challenge with direct implications for personalized immunotherapy, disease monitoring, and vaccine response studies. By analyzing the sequences generated by each model, we identify memorization risks, innovation capabilities, and sensitivity to hyperparameter tuning. Taken together, these results advance the understanding of how current generative models reproduce the biology of individual immune repertoires and lay the groundwork for more principled model development and evaluation.