With a Little Help from AI: Moral Orientation, Trust in AI, and Acceptance of AI-Based Advice Utilization in Moral Dilemmas Involving Partially Automated Car

Ethical concerns surrounding self-driving cars persist, particularly in crash scenarios. In previous studies, participants judged the moral permissibility of actions in self-driving car dilemmas without interacting with AI or revisiting their initial judgments. However, resolving moral dilemmas could be a collaborative effort between human users and AI, as recent technological advances emphasize human-AI interaction for enhanced synergy. Using the judge-advisor paradigm, this study examined contributions of moral orientation, trust in AI, and empathic concern to predicting advice taking from AI in self-driving car moral dilemmas. Participants were presented with an AI advice that contradicted with their initial judgment. The weight of advice (WOA) indicated the extent to which participants took advice from AI, where WOA = 1 equals to a complete reliance on the advice. Three dilemmas varied in the presence of self-sacrificial options and outcomes (non-utilitarian vs. utilitarian). Results show that 26.6–34.8% of drivers may take advice from AI even when it disagrees with their initial judgment. Moral orientation was associated with self-protective or other-oriented advice-taking, depending on the situation. Additionally, participants with high empathy were more likely to adopt AI’s self-sacrificial suggestion to save more people. These findings suggest that moral principles shift based on the situation and outcomes, highlighting the challenge of implementing AI systems with fixed moral programming in situations where moral principles conflict. Furthermore, individual variations in the acceptability of AI advice indicate that questions of responsibility attribution also arise, potentially creating divisions in the absence of clear guidelines.