Measures of Argument Strength: A Computational, Large-Scale Analysis of Effective Persuasion in Real-World Debates

The present research examined how value-free and value-driven measures of argument strength (MAS) can be computationally extracted using a theory-driven approach at scale in a naturalistic setting by analyzing a total of 7,961 real-world debates and 42,716 judgments in rhetorical quality. In the first study, value-free MAS was significantly related to the rhetorical quality of arguments (i.e. their persuasiveness). The results indicate that the side that provides more information-source citation, less quantitative specificity, more unique words, and more abstract language is more likely to be perceived as convincing in dialectical argumentation, where two people are exchanging opposing arguments. In the second study, the added influence of value-driven MAS is investigated. The results show that the similarity between the moral values represented in arguments and those that are salient to argument receivers predicts the rhetorical quality. The research demonstrates how rhetorical quality can be measured and predicted at scale, and how naturally generated arguments can be used for scientific progress in persuasion research.