Improving Adaptive Pairing Method in Incomplete PairedComparison Design

We use paired comparison to rank items. If the number of items is large, we have to compare them many times. In such a case, implementation of an experiment becomes infeasible because of the expense and time required for each experimental run. In addition, evaluation criteria vary from person to person. Therefore, we need experimental designs to take these factors into account. In this paper, we discuss the experimental designs of paired comparisons in incomplete designs. We compare three pairing rules, namely, slide pairing, fold pairing, and adjacent pairing. As a result, we find that adjacent pairing is inferior to slide pairing and fold pairing. It is necessary to select slide pairing or fold pairing based on the number of comparisons. However, after a threshold number of comparisons, slide pairing and fold pairing do not always yield the best results. These results clarify the recommended number of comparisons and the best pairing rules. We believe that the results can be used by people who use paired comparison in practice.