Abstract Background Magnesium’s (Mg) role in brain aging is under-studied. We examined associations of dietary and serum Mg with MRI and cognitive measures in a cohort with >10 y follow-up. Methods Adults from the Boston Puerto Rican Health Study who underwent MRI (n = 218 with serum Mg, n = 198 with Mg intake) and cognitive testing (n = 1049 with serum Mg, n = 1363 with Mg intake) were included. Hypomagnesemia was defined as serum Mg <0.75 mmol/L, and adequate Mg intake as ≥ the Estimated Average Requirement (males: 350 mg/d, females: 265 mg/d). Multivariable linear regression and linear mixed-effects models were used to estimate associations of Mg with brain age gap (z-score), volumes (% of total intracranial volume), and cognitive scores (z-score). Effect modification by diabetes, hypertension, dyslipidemia, calcium: Mg intake ratio, or ApoE4 was evaluated. Results In the full sample, hypomagnesemia was not associated with brain MRI measures, but with global cognitive function [β (95% CI) = −0.058 (−0.11, −0.0020)], in fully adjusted models. Adequate Mg intake was associated with volumes of total gray matter [β (95% CI) = 2.83 (0.56, 5.11)], white matter [β (95% CI) = 1.54 (0.22, 2.86)], and multiple subregions, but not cognition. In participants with diabetes, hypomagnesemia was associated with lower volumes of multiple brain regions, which were not observed in participants without diabetes (pinteraction < .05). Limited or no interaction was detected for other potential modifiers, despite some variation in association direction or magnitude across subgroups. Conclusions Diabetes modified Mg’s association with brain volumes. Mg merits more attention among people with diabetes for preventing neurodegeneration.