A comment on "An Empirical Investigation of the Impact of ChatGPT on Creativity"

作者
Prochazka, Jakub,Zhou, Jing,Coita, Ioana-Florina,Akhtar, Shumi
出处
期刊:Econstor - Econstor
摘要

This report describes a computational reproduction of Lee and Chung's (2024) paper, which examined whether using ChatGPT (GPT‑3.5) enhances creativity in adults compared to web-search assistance or no assistance. The authors presented six randomized controlled experiments showing that ChatGPT-assisted responses were assessed as significantly more creative (effect sizes ranging from Cohen's d = 0.32 to 1.88). These effects were robust across diverse tasks and contexts. We first computationally reproduced all the main results using the original dataset and code, obtaining the same results as those presented by the authors in their paper. During the reproduction process, we identified two minor coding errors and one typographical error in the original table, none of which affected the substantive conclusions. Second, we performed a recreate reproduction for the main analysis in Experiments 1 and 3 by writing new R code. Our results again matched the results presented in the original paper. Overall, based on our analyses, the study is fully computationally reproducible from raw data, although only with access to the original code, due to undocumented cleaning steps, some non-described exclusion criteria, and missing codebooks. Several analyses in the original paper showed that ideas generated by ChatGPT are rated as similarly creative regardless of whether people modify them or not. We contributed to this conclusion by introducing a new robustness check using response time as a proxy for human effort in modifying ChatGPT outputs. Using data from Experiment 3, we found no significant correlation between response time and creativity in the ChatGPT condition (r = −.079, p = .449) and no moderating effect of response time on the influence of using ChatGPT on creativity. This suggests that human effort does not incrementally improve creativity beyond ChatGPT's contribution. Taken together, our findings support the original claim that using ChatGPT increases creativity regardless of the human input.

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
Wzebrafish完成签到,获得积分10
1秒前
Ava的应助被雨笙采纳,获得10
2秒前
一个薯片完成签到,获得积分10
2秒前
yys发布了新的文献求助10
3秒前
完美的念梦完成签到,获得积分10
3秒前
Sledge发布了新的文献求助10
4秒前
5秒前
5秒前
5秒前
daomaihu发布了新的文献求助100
5秒前
迅速的柚子完成签到,获得积分10
6秒前
张奕泽完成签到 ,获得积分10
6秒前
眯眯眼的沧海完成签到,获得积分20
7秒前
7秒前
科研通AI65的应助被Czz采纳,获得10
7秒前
一期一会完成签到,获得积分10
8秒前
8秒前
领导范儿的应助被韦远山采纳,获得10
8秒前
8秒前
顾矜的应助被机智的三国菌采纳,获得10
8秒前
泉汀野和完成签到,获得积分10
9秒前
戴上耳机去赶羊完成签到,获得积分10
9秒前
莫小乖发布了新的文献求助10
9秒前
Mr权发布了新的文献求助20
9秒前
fan完成签到,获得积分10
9秒前
YuJue的应助被zz采纳,获得50
10秒前
10秒前
11秒前
11秒前
杰_骜不驯完成签到,获得积分10
13秒前
优美聪健完成签到 ,获得积分10
13秒前
moyu完成签到,获得积分10
13秒前
小蛋完成签到,获得积分20
13秒前
DFILY完成签到,获得积分10
13秒前
Kannan发布了新的文献求助10
14秒前
电子屎壳郎完成签到,获得积分10
14秒前
CHEN完成签到,获得积分20
15秒前
15秒前
iveuplife发布了新的文献求助10
16秒前
888完成签到,获得积分10
16秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
John McDowells Bildungstheorie 800
八旗制度の研究 600
plantTFDB 508
Resource Recovery Handbook 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8175336
求助须知:如何正确求助?哪些是违规求助? 9567158
关于积分的说明 21790491
捐赠科研通 7769866
什么是DOI,文献DOI怎么找? 3431849
关于科研通互助平台的介绍 2556178
邀请新用户注册赠送积分活动 2477280