BrainSeg: a generalized framework for comprehensive multimodal brain tissue segmentation, parcellation, and lesion labeling

Precise brain segmentation is fundamental for quantitative neuroimaging analysis. However, most existing methods lack generalization across the human lifespan and diverse imaging modalities, limiting their utility for Comprehensive Brain Segmentation (CBS) (i.e., tissue segmentation, parcellation, and lesion labeling). To address this, we propose BrainSeg, a novel unified framework, for CBS by using large-scale datasets spanning the entire lifespan, with adaptability to diverse uni- and multimodal input scenarios without the need for retraining or finetuning. Comprehensive experiments are conducted on lifespan data ranging from 14 gestational weeks to 100 years of age, consisting of 45,998 multimodal scans from 26 datasets, which are further augmented by our proposed synthesis strategy. Systematic validation and in-depth analysis demonstrate that our BrainSeg can achieve state-of-the-art performance across all three core CBS tasks, with the averaged Dice ratios reaching up to 96.94% for tissue segmentation, 94.25% for brain parcellation, and 91.06% for lesion labeling in the internal validations. It maintains similarly high accuracy in external validations, with averaged Dice ratios achieving 94.01% for tissue segmentation, and 91.20% for brain parcellation, underscoring its robustness and generalizability across diverse conditions. In summary, BrainSeg serves as a versatile foundation tool, providing flexible and reliable analysis for large-scale neuroimaging studies.