The Power of Disagreement: A Field Experiment to Investigate Human–Algorithm Collaboration in Loan Evaluations

Human-algorithm collaboration is becoming increasingly prevalent in the economy and society. However, this collaboration is not always fruitful, and in extreme cases people become human borgs or totally averse to algorithms. The key to collaborative value is whether humans and algorithms can complement each other in decision-making, but it is challenging for humans to disagree with algorithmic recommendations at the right time (i.e., to disagree when algorithms are wrong and not disagree when algorithms are right). To understand the centric role of disagreement in human-algorithm collaboration and examine when and how it benefits, we conducted a field experiment where human evaluators and algorithms worked together to evaluate loan applications under four scenarios, i.e., limited/rich information and with/without disclosure of algorithm rationale. Our results showed that human-algorithm collaboration decisions outperformed human-or algorithm-only decisions, and these collaborative values varied across four scenarios. We further proposed a theoretical framework for mechanism examination that centered on the formation and effectiveness of disagreement. We validated the framework empirically and came to the following findings: (1) disagreement exhibited a sizable and non-linear influence on collaborative value; (2) the differences between human evaluators and algorithms in decision-making contributed to disagreement but not to collaborative value; (3) algorithm self-contradiction triggered disagreement and helped human evaluators disagree with algorithms at the right time. These findings provide valuable theoretical insights on how collaborative value is achieved and managerial insights on how to manage disagreement in humanalgorithm collaboration.