Writing with emotion? Assessing emotional valence and appeals in AI-generated vs. human-written articles

Abstract Does generative AI write with emotionality in the same manners as humans? We investigated this question by focusing on three distinct aspects that characterize emotionally evocative content, including emotional valence, discrete appeals, and subjectivity. A quantitative content analysis of two publicly available data sets—TOEFL essays vs. news articles—was conducted, which paired each AI-generated text with a human-written piece by matching the essay questions or news headlines. Results revealed a stronger level of subjectivity in AI-generated than human-written articles. We also found the differences between AI-generated and human-written texts may differ by the type of writing: in terms of emotional valence, while little variance existed in TOEFL essays, AI-generated news stories exhibited a stronger positivity than human-written news. Similarly, regarding discrete emotional appeals, notable differences were identified between AI and human among news stories but not among TOEFL essays. In particular, AI-generated soft news is more likely convey a sadness emotion than human counterparts. These findings suggest that both the level and discrete states of emotionality are significant factors for studying the impact of AI-generated content on readers.