Assessing Agreement Between Two Methods of Quantitative Measurements: Exact Test Procedure and Sample Size Calculation

Assessment of agreement between measurement methods is an important problem in medical practice and research. Several two one-sided tests (TOST) procedures have been proposed to evaluate whether two methods have sufficient agreement. These TOST techniques of limits of agreement or normal percentiles are natural generalizations of the favorable TOST procedure of mean differences. But their fundamental behavior of Type I error control has not been examined. This article describes an improved and exact approach for declaring agreement between two measurement methods when the range of central proportion of paired differences is within the designated threshold interval around zero. Theoretical examinations and numerical investigations are provided to justify the advantages of the proposed approach and to illustrate the shortcomings of the existing TOST procedures for assessing agreement between two methods. To facilitate the suggested agreement appraisal in design planning, the corresponding power and sample size formulas are also derived. Both SAS and R algorithms are presented for conducting agreement test, power calculation, and sample size determination in method comparison studies.