Do Crowds Validate False Data? Systematic Distortion and Affective Polarization

This research note examines how sociocognitive influences can systematically distort crowdsourced ground truth in event-centric data through subgroups. The “wisdom of the crowd” is based on the assumption that consensus drives accuracy. While existing research addresses the tendencies of the overall crowd, this research note shows that identifiable subgroups within the crowd can systematically influence crowdsource validation. We conducted an immersive experiment to investigate whether crowd consensus can be systematically distorted by subgroup-based sociocognitive influences, such as affective polarization. In the experiment, raters from a range of subgroups with varying levels of affective polarization were asked to view and validate crisis data from a violent public riot in the year 2020. Relying in part on double debiased machine learning techniques, we analyzed heterogeneous treatment effects across subgroups. The results show that affective polarization and more extreme raters, via the constructs of loyalty and betrayal, distort consensus-based ground truth in different ways. This research note demonstrates how subgroup-based sociocognitive influences can systematically distort the results of consensus-based crowdsourced validation. Additionally, it provides guidance for research and practice on how to account for identifiable subgroups in the crowd. These findings challenge key assumptions about the wisdom of crowds and the accuracy of crowdsourced ground truth in event-centric situations.