ABSTRACT Background Cognitive decline commonly co‐occurs with dynamic physical and mental health changes in older adults. While early‐life adversity has been linked to various later‐life health outcomes, its relationship with cognitive function considering different health trajectories remains unclear. Objectives To identify distinct clusters of physical‐mental‐cognitive health trajectories among Chinese older adults and to examine the association between childhood adversities and cognitive function across these trajectories. Methods Using data from the China Health and Retirement Longitudinal Study (2011–2020), we included 6178 adults aged ≥ 60 years. Latent Class Growth Modeling was used to identify trajectory patterns of functional limitations, depressive symptoms, and cognitive function. Mixed linear models examined associations between childhood adversities and cognitive function overall and across different identified trajectory patterns. Results Four distinct trajectory classes were identified: healthy individuals (59.8%), rapid cognitive decline with gradual physical‐mental decline (16.5%), mild cognitive decline with physical‐mental improvement (14.4%), and moderate cognitive decline with rapid physical and moderate mental decline (9.4%). Experience of multiple childhood adversities was significantly associated with lower cognitive function ( β = −0.36, 95% CI [−0.58, −0.14]), independent of adulthood factors and consistent across various trajectory patterns. Among individuals showing rapid cognitive decline with gradual physical‐mental deterioration, experiencing two childhood adversities predicted lower cognitive function ( β = −0.88, 95% CI [−1.62, −0.14]). Conclusion Childhood adversities are associated with cognitive impairment regardless of physical‐mental‐cognitive health trajectories in older Chinese adults. These findings highlight the long‐term impact of early‐life experiences on cognitive health in later life.