Assessing Moral Decision Making in Large Language Models
作者
Chris Shaner,Henry Griffith,Heena Rathore
出处
期刊:International Conference on Consumer Electronics [Institute of Electrical and Electronics Engineers] 日期:2025-01-11卷期号:: 1-3被引量:2
标识
DOI:10.1109/icce63647.2025.10930088
摘要
As large language models (LLMs) increasingly permeate diverse industries, it has become crucial to comprehend the mechanisms underlying their moral decision-making processes. Existing studies on the moral decision-making processes of LLMs have been limited in their evaluation of ethical theories. The datasets utilized in these studies relies on human preferences, which introduces a set of inherent flaws. This paper utilizes different prompting styles to assess LLMs on moral ethical questions, addressing challenges associated with human preferences. The contributions include a dataset with human preferences and LLM preference on diverse ethical theories namely commonsense, virtue ethics, justice and deontology. We scrutinize these ethical decision-making on GPT-4 Bard, Mistral, and LLaMA2 to create a broad understanding of ethical reasoning capabilities across different platforms. We used baselines metrics such as accuracy, and F1-score for the evaluation purposes. GPT4 demonstrated a remarkable alignment with human ethical judgments, achieving an average accuracy of 81.5%. By including AI-generated justifications for ethical decisions, our study not only enhances the transparency of AI decision-making but also provides a foundational understanding of how LLMs conceptualize and apply ethical principles.