When Algorithmic Predictions Use Human-Generated Data: A Bias-Aware Classification Algorithm for Breast Cancer Diagnosis

When algorithms use data generated by human beings, they inherit the errors stemming from human biases, which likely diminishes their performance. We examine the design and value of a bias-aware linear classification algorithm that accounts for bias in input data, using breast cancer diagnosis as our specific setting. In this context, a referring physician makes a follow-up recommendation to a patient based on two inputs: the patient’s clinical-risk information and the radiologist’s mammogram assessment. Critically, the radiologist’s assessment could be biased by the clinical-risk information, which in turn can negatively affect the referring physician’s performance. Thus, a bias-aware algorithm has the potential to be of significant value if integrated into a clinical decision support system used by the referring physician. We develop and show that a bias-aware algorithm can eliminate the adverse impact of bias if the error in the mammogram assessment due to radiologist’s bias has no variance. On the other hand, in the presence of error variance, the adverse impact of bias can be mitigated, but not eliminated, by the bias-aware algorithm. The bias-aware algorithm assigns less (more) weight to the clinical-risk information (radiologist’s mammogram assessment) when the mean error increases (decreases), but the reverse happens when the error variance increases. Using point estimates obtained from mammography practice and the medical literature, we show that the bias-aware algorithm can significantly improve the expected patient life years or the accuracy of decisions based on mammography. The online appendix is available at https://doi.org/10.1287/isre.2018.0789 .