Doing Justice to Logical Remainders in QCA: Moving beyond the Standard Analysis

AbstractLimited diversity is among the most understudied methodological challenges. QCA allows for a more conscious treatment of logical remainders than most other comparative methods. The current state of the art is the Standard Analysis (Ragin 2008; Ragin and Sonnett 2004). We discuss two of its pitfalls, both rooted in the primacy given to parsimony. First, the Standard Analysis is no safeguard against untenable assumptions. As a remedy, we propose the Enhanced Standard Analysis (ESA). Second, researchers should consider including theoretically sound counterfactual claims even if they do not contribute to parsimony. We label this Theory-Guided Enhanced Standard Analysis (TESA).IntroductionArguably, limited diversity is one of the most understudied phenomena in comparative social research methodology. It refers to situations in which one or more logically possible combinations of conditions are void of (enough) empirical evidence. In set-theoretic methods, such combinations are labeled logical remainders.1 Limited diversity is omnipresent in empirical comparative research, a fact that is often overlooked because most data analysis techniques make it difficult to see logical remainders when they are there. This is a severe shortcoming, for limited diversity undermines the possibilities of drawing valid inferences. With QCA, due to the central role truth tables play in this technique, limited diversity is easy to detect.2 Drawing attention to this problem and providing innovative solutions can certainly be seen as one of the most distinct contributions of QCA over the past twenty-five years.3Currently, the state-of-the-art in tackling limited diversity is the so-called Standard Analysis (Ragin 2008; Ragin and Sonnett 2004). It consists of producing (a) the complex solution, or what we prefer to call the conservative solution,4 where no assumption about any logical remainder is made; (b) the most parsimonious solution,5 which is based on simplifying assumptions, that is, those remainders are included into the logical minimization that contribute to parsimony; and (c) the intermediate solution, which relies on easy counterfactuals, that is, only those simplifying assumptions are included that are in line with theory-driven directional expectations.The intermediate solution has several advantageous properties that should make it the focus of substantive interpretation (Ragin 2008, chap. 9). First, it is in between the conservative and the most parsimonious solution in terms of complexity. The former often tends to be too complex for a theoretically meaningful interpretation, and the latter is often too parsimonious and risks resting on assumptions about logical remainders that are difficult to sustain based on theoretical grounds. Second, the intermediate solution is-by definition-a subset of the most parsimonious solution and a superset of the conservative solution. This directly follows from the requirement that easy counterfactuals can only be chosen from those simplifying assumptions that have already been used for deriving the most parsimonious solution.In applied QCA, truth tables often contain a large number of logical remainders. The range of logically possible solutions, created through assumptions on different remainders, is therefore often large. The intermediate solution of the Standard Analysis is an appealing tool for handling this issue, because it (often drastically) limits the number of eligible remainder rows. It does so by providing a protocol for selecting logical remainders for counterfactual claims by, first, identifying those remainders that produce simplifying assumptions and, second, by selecting among them those that are in line with theoretical expectations about single conditions. While in Step 1, both difficult and easy counterfactuals are considered, in Step 2, the latter are separated from the former.6This article is motivated by two observations. First, in applied QCA, the proper treatment of logical remainders is still often taken too lightly (Mendel and Ragin 2011, 10). …