心理学
归属
认知心理学
感知
认知
社会心理学
社会认知
聚类分析
计算机科学
方向(向量空间)
内容分析
内容(测量理论)
认知语言学
领域(数学分析)
归因偏差
自然语言处理
因果关系(物理学)
人工智能
社会认知
认知偏差
召回
生物学与政治取向
作者
Alexander Loth,Martin Kappes,Marc‐Oliver Pahl
出处
期刊:
[Association for Computing Machinery]
日期:2026-05-23
卷期号:: 34-39
被引量:3
标识
DOI:10.1145/3795513.3807431
摘要
Can humans tell whether a news article was written by a person or a large language model (LLM)? We investigate this question using JudgeGPT, a study platform that independently measures source attribution (human vs. machine) and authenticity judgment (legitimate vs. fake) on continuous scales. From 2,318 judgments collected from 1,054 participants across content generated by six LLMs, we report five findings: (1) participants cannot reliably distinguish machine-generated from human-written text (p >.05, Welch’s t-test); (2) this inability holds across all tested models, including open-weight models with as few as 7B parameters; (3) self-reported domain expertise predicts judgment accuracy (r =.35, p <.001) whereas political orientation does not (r = −.10, n.s.); (4) clustering reveals distinct response strategies (“Skeptics” vs. “Believers”); and (5) accuracy degrades after approximately 30 sequential evaluations due to cognitive fatigue. The answer, in short, is no: humans cannot reliably tell. These results indicate that user-side detection is not a viable defense and motivate system-level countermeasures such as cryptographic content provenance.
科研通智能强力驱动
Strongly Powered by AbleSci AI