On the Importance of Providing a “Don’t Know” Response Option in Syllogistic Reasoning Tasks

Abstract: Syllogistic reasoning tasks typically contain a response option stating that no valid conclusion can be drawn from the information. This response is the logically correct response for the majority of all syllogisms. However, the same response may also be chosen by participants who are simply unable to find the correct solution. In the current study, we suggest using an additional “don’t know” response option so that unsuccessfully aborted reasoning processes are not systematically classified as successful. We demonstrate that, when given the opportunity, participants make systematic use of a “don’t know” response option. Further, the inclusion and use of the response option significantly impacted participants’ response distribution when compared to a group of participants that did not have such a response option. Our findings suggest that without an opt-out option, participants engage in biased guessing behavior when they “give-up” on a problem, leading to incorrect ability estimates in testing situations. We conclude that future tests employing syllogistic reasoning tasks should include an additional “don’t know” response option to obtain unbiased measures of reasoning abilities.