10-year performance of four models of breast cancer risk: a validation study

Independent validation is essential to justify use of models of breast cancer risk prediction and inform decisions about prevention options and screening. Few independent validations had been done using cohorts for common breast cancer risk prediction models, and those that have been done had small sample sizes and short follow-up periods, and used earlier versions of the prediction tools. We aimed to validate the relative performance of four commonly used models of breast cancer risk and assess the effect of limited data input on each one's performance.In this validation study, we used the Breast Cancer Prospective Family Study Cohort (ProF-SC), which includes 18 856 women from Australia, Canada, and the USA who did not have breast cancer at recruitment, between March 17, 1992, and June 29, 2011. We selected women from the cohort who were 20-70 years old and had no previous history of bilateral prophylactic mastectomy or ovarian cancer, at least 2 months of follow-up data, and information available about family history of breast cancer. We used this selected cohort to calculate 10-year risk scores and compare four models of breast cancer risk prediction: the Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm model (BOADICEA), BRCAPRO, the Breast Cancer Risk Assessment Tool (BCRAT), and the International Breast Cancer Intervention Study model (IBIS). We compared model calibration based on the ratio of the expected number of breast cancer cases to the observed number of breast cancer cases in the cohort, and on the basis of their discriminatory ability to separate those who will and will not have breast cancer diagnosed within 10 years as measured with the concordance statistic (C-statistic). We did subgroup analyses to compare the performance of the models at 10 years in BRCA1 or BRCA2 mutation carriers (ie, BRCA-positive women), tested non-carriers and untested participants (ie, BRCA-negative women), and participants younger than 50 years at recruitment. We also assessed the effect that limited data input (eg, restriction of the amount of family history and non-genetic information included) had on the models' performance.After median follow-up of 11·1 years (IQR 6·0-14·4), 619 (4%) of 15 732 women selected from the ProF-SC cohort study were prospectively diagnosed with breast cancer after recruitment, of whom 519 (84%) had histologically confirmed disease. BOADICEA and IBIS were well calibrated in the overall validation cohort, whereas BRCAPRO and BCRAT underpredicted risk (ratio of expected cases to observed cases 1·05 [95% CI 0·97-1·14] for BOADICEA, 1·03 [0·96-1·12] for IBIS, 0·59 [0·55-0·64] for BRCAPRO, and 0·79 [0·73-0·85] for BRCAT). The estimated C-statistics for the complete validation cohort were 0·70 (95% CI 0·68-0·72) for BOADICEA, 0·71 (0·69-0·73) for IBIS, 0·68 (0·65-0·70) for BRCAPRO, and 0·60 (0·58-0·62) for BCRAT. In subgroup analyses by BRCA mutation status, the ratio of expected to observed cases for BRCA-negative women was 1·02 (95% CI 0·93-1·12) for BOADICEA, 1·00 (0·92-1·10) for IBIS, 0·53 (0·49-0·58) for BRCAPRO, and 0·97 (0·89-1·06) for BCRAT. For BRCA-positive participants, BOADICEA and IBIS were well calibrated, but BRCAPRO underpredicted risk (ratio of expected to observed cases 1·17 [95% CI 0·99-1·38] for BOADICEA, 1·14 [0·96-1·35] for IBIS, and 0·80 [0·68-0·95] for BRCAPRO). We noted similar patterns of calibration for women younger than 50 years at recruitment. Finally, BOADICEA and IBIS predictive scores were not appreciably affected by limiting input data to family history for first-degree and second-degree relatives.Our results suggest that models that include multigenerational family history, such as BOADICEA and IBIS, have better ability to predict breast cancer risk, even for women at average or below-average risk of breast cancer. Although BOADICEA and IBIS performed similarly, further improvements in the accuracy of predictions could be possible with hybrid models that incorporate the polygenic risk component of BOADICEA and the non-family-history risk factors included in IBIS.US National Institutes of Health, National Cancer Institute, Breast Cancer Research Foundation, Australian National Health and Medical Research Council, Victorian Health Promotion Foundation, Victorian Breast Cancer Research Consortium, Cancer Australia, National Breast Cancer Foundation, Queensland Cancer Fund, Cancer Councils of New South Wales, Victoria, Tasmania, and South Australia, and Cancer Foundation of Western Australia.