The Emerging Hazard of AI‐Related Health Care Discrimination

Abstract Artificial intelligence holds great promise for improved health‐care outcomes. But it also poses substantial new hazards, including algorithmic discrimination. For example, an algorithm used to identify candidates for beneficial “high risk care management” programs routinely failed to select racial minorities. Furthermore, some algorithms deliberately adjust for race in ways that divert resources away from minority patients. To illustrate, algorithms have underestimated African Americans’ risks of kidney stones and death from heart failure . Algorithmic discrimination can violate Title VI of the Civil Rights Act and Section 1557 of the Affordable Care Act when it unjustifiably disadvantages underserved populations. This article urges that both legal and technical tools be deployed to promote AI fairness. Plaintiffs should be able to assert disparate impact claims in health‐care litigation, and Congress should enact an Algorithmic Accountability Act. In addition, fairness should be a key element in designing, implementing, validating, and employing AI .