Selecting and interpreting measures of thematic classification accuracy

Abstract An error matrix is frequently employed to organize and display information used to assess the thematic accuracy of a land-cover map, and numerous accuracy measures have been proposed for summarizing the information contained in this error matrix. No one measure is universally best for all accuracy assessment objectives, and different accuracy measures may lead to conflicting conclusions because the measures do not represent accuracy in the same tray. Choosing appropriate accuracy measures that address objectives of the mapping project is critical. Characteristics of some commonly used accuracy measures are described, and relationships among these measures are provided to aid the user in choosing an appropriate measure. Accuracy measures that are directly interpretable as probabilities of encountering certain types of misclassification errors or correct classifications should be selected in. preference to measures not interpretable as such. User's and producer's accuracy and the overall proportion OF area correctly classified are examples of accuracy measures possessing the desired probabilistic interpretation. The kappa coefficient of agreement does not possess such a probabilistic interpretation because of the adjustment for hypothetical chance agreement incorporated into this measure, and the strong dependence of kappa on the marginal proportions of the error matrix makes the utility of kappa for comparisons suspect. Normalizing an error matrix results in estimates that are not consistent for accuracy paramteters of the map being assessed, so that this procedure is generally not warranted for most applications.