Living with Imperfect AI: User Adaptation, Trust Recalibration, and the Emerging Burden of Verification in Response to Generative AI Hallucinations
作者
Rongqin Liu,Fangying Pang,Hui Jiang,Zhiyong Cai,Min Zhang,Lili Li
标识
DOI:10.1145/3833541.3833828
摘要
Generative AI systems are increasingly embedded in knowledge work, yet their tendency to produce plausible but incorrect outputs—commonly referred to as hallucinations—poses persistent challenges. While prior research has primarily focused on technical mitigation, less is known about how users adapt to and manage these failures in practice. This paper presents a mixed-method study combining qualitative interviews with computational text analysis, based on 22 semi-structured interviews with active users of generative AI tools. Through inductive thematic analysis complemented by topic modeling, semantic clustering, and sentiment analysis, we examine how users interpret hallucinations, how these experiences reshape trust, and how workflows evolve in response. We find that encounters with salient hallucination incidents act as critical turning points that disrupt initial over-reliance and lead to more differentiated and task-contingent trust. Users develop a pragmatic mental model of AI as optimizing for plausibility rather than factual accuracy, and adopt a set of adaptive strategies, including systematic verification practices, task triaging, and prompt-level control. Building on these findings, we introduce the concept of the burden of verification to describe the additional cognitive, temporal, and affective work required to safely integrate AI into professional workflows. Rather than treating hallucinations solely as technical failures, we argue they should be understood as a socio-technical condition that reshapes human-AI collaboration. Our work contributes empirical insights into user adaptation in the presence of unreliable AI and demonstrates how computational text analysis can be used to support and extend qualitative findings in human-AI interaction research.