Not Good Enough, but Try Again! The Impact of Improved Rejection Communications on Contributor Retention and Performance in Open Knowledge Collaboration

Producing high-quality content through open knowledge collaboration presents a dilemma. Successful quality control requires rejecting contributions that do not align with the aims of an open knowledge collaboration system, yet rejections demotivate new contributors from attempting further contributions. Building on research on the regulation of behavior in online communities and on organizational selection, we theorize how the way rejections are communicated affect new contributors. We leverage a change to the rejection notices used by the Stack Overflow community question-answering service to study the issue empirically. First, we use a regression discontinuity in time design to construct a natural experiment to estimate the average treatment effect of more informative rejection notices on the retention of initially rejected contributors. The results show that notices that better explain the reason for rejecting a contributor’s initial question—and therefore reduce uncertainty about the outcome if the contributor tries again—increase retention by approximately 21.7 percentage points. Second, we use a mediation model to study mechanisms by which more informative rejection notices affect contributor performance. We find that more informative notices affect contributor performance through selection rather than performance improvement. Additional contributors, who are retained because of the more informative rejection notices, ask more questions, on average, whereas we find no evidence of individual contributors improving their performance in terms of the quality or quantity of their contributions. Overall, the results suggest that rejection communications offer low-cost interventions to mitigate the trade-off between new contributor retention and the quality of contributions in open knowledge collaboration. This paper was accepted by Hemant Bhargava, information systems. Funding: Financial support from the Fox School of Business, Temple University (Young Scholar Interdisciplinary Forum) is gratefully acknowledged. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2021.02720 .