Reproducibility studies in Manual/Musculoskeletal Medicine: a new method for kappa independence from prevalence

In M/M Medicine, it has become increasingly important that diagnostic tests are reproducible. The kappa statistic is the measure most frequently used to define the interobserver agreement of diagnostic procedures. The main disadvantage of the kappa statistic is its dependence on the prevalence, making a good kappa value at the end of every reproducibility study always unpredictable. A previous published theoretical protocol proposed solving this problem by obtaining a prevalence near 0.50. This was evaluated in the present study of the passive hip flexion test. A prevalence of 0.44 was found with a good to excellent kappa value of 0.75. It is concluded that when implementing the proposed method in the protocol format for reproducibility studies, using kappa statistics, a prevalence P near 0.50 can easily be obtained avoiding unexpected low kappa values.