Quantifying Parkinson’s disease motor severity under uncertainty using MDS-UPDRS videos

• We propose a generic pipeline for estimating movement impairment severity scores using body or hand skeletons and classify them into clinical scores. • We assess the inter-rater reliability of multiple ratings from 3 different raters and propose a pipeline to learn clinical score estimation under uncertainty. • We extend our model via the Rater Confusion Estimation framework trained by our novel ordinal focal loss, with the addition of an explicit simplex projection for learning. • We present saliency visualizations to stratify the contribution of separate body joints to the estimation of MDS-UPDRS scores. Parkinson’s disease (PD) is a brain disorder that primarily affects motor function, leading to slow movement, tremor, and stiffness, as well as postural instability and difficulty with walking/balance. The severity of PD motor impairments is clinically assessed by part III of the Movement Disorder Society Unified Parkinson’s Disease Rating Scale (MDS-UPDRS), a universally-accepted rating scale. However, experts often disagree on the exact scoring of individuals. In the presence of label noise, training a machine learning model using only scores from a single rater may introduce bias, while training models with multiple noisy ratings is a challenging task due to the inter-rater variabilities. In this paper, we introduce an ordinal focal neural network to estimate the MDS-UPDRS scores from input videos, to leverage the ordinal nature of MDS-UPDRS scores and combat class imbalance. To handle multiple noisy labels per exam, the training of the network is regularized via rater confusion estimation (RCE), which encodes the rating habits and skills of raters via a confusion matrix. We apply our pipeline to estimate MDS-UPDRS test scores from their video recordings including gait (with multiple R aters, R = 3 ) and finger tapping scores (single rater). On a sizable clinical dataset for the gait test ( N = 55 ), we obtained a classification accuracy of 72% with majority vote as ground-truth, and an accuracy of ∼ 84% of our model predicting at least one of the raters’ scores. Our work demonstrates how computer-assisted technologies can be used to track patients and their motor impairments, even when there is uncertainty in the clinical ratings. The latest version of the code will be available at https://github.com/mlu355/PD-Motor-Severity-Estimation .