Abstract Study Objectives The long-term trajectory of cognitive changes in isolated REM sleep behavior disorder (iRBD) remains unclear. Increasing evidence suggests potential heterogeneity in disease progression. Methods Patients who maintained an iRBD diagnosis without phenoconversion for at least five years were included. Cognitive changes over time were assessed using linear mixed models. Subgroup analyses were conducted for men and women separately, and for longstanding iRBD patients. Results A total of 318 neuropsychological assessments from 162 patients with iRBD (28% women) were analyzed. The mean age at diagnosis was 65.6 years, with an average follow-up duration of 7.7 years. Significant linear declines were observed across attention/working memory and memory domains, with notable decreases in Digit Symbol ($\boldsymbol{\beta}$= −0.084, 95% confidence intervals (CI) −0.102 to −0.066), Stroop Test color naming ($\boldsymbol{\beta}$= −0.054, 95% CI −0.074 to −0.054), and Word List Recall ($\boldsymbol{\beta}$= −0.054, 95% CI −0.078 to −0.030) as representative examples. In the longstanding iRBD subgroup (n = 33) and among men, cognitive decline patterns largely mirrored those observed in the full cohort, with significant declines in attention/working memory and memory. In contrast, women exhibited a more restricted decline, limited to Digit Span Forward ($\boldsymbol{\beta}$= −0.055, 95% CI −0.102 to −0.008) and Digit Symbol ($\boldsymbol{\beta}$= −0.082, 95% CI −0.121 to −0.043). Conclusions Gradual cognitive decline in attention/working memory and memory may represent a natural course of neurodegeneration in men with iRBD, without necessarily indicating imminent phenoconversion. Women with iRBD appeared to show greater resilience to cognitive decline compared to men.