Background: Individuals with atrial fibrillation (AF) are more likely to develop mild cognitive impairment (MCI), but the underlying mechanisms remain unclear. The study aimed to investigate cognitive-related gray matter (GM) volume alterations in stroke-free individuals with AF using voxel-based morphometry (VBM). Methods: 3D-T1-weighted magnetic resonance imaging (MRI) scans were obtained from 40 stroke-free AF individuals with MCI (AF-MCI), 40 stroke-free AF individuals with normal cognition (AF-NC), and 40 healthy controls (HCs). GM atrophy was assessed using VBM. Results: The results revealed widespread GM atrophy in stroke-free individuals with AF, regardless of their cognitive status, with more pronounced GM loss in the AF-MCI group. Significant GM volume reductions were found in several brain regions, including the temporal lobe, parahippocampal gyrus (PHG), cerebellum, and frontal lobe, in the AF-MCI group. Notable reductions in the left PHG and right inferior parietal lobule were observed in the AF-MCI group compared with the AF-NC group. Moreover, decreased GM volume in the left PHG, right superior temporal pole, and right orbital part of the inferior frontal gyrus was positively correlated with cognitive performance. Conclusions: Among AF individuals free of stroke, degeneration of the PHG correlates with a greater probability of developing MCI. Structural alterations in the brain may be related to the transition from normal cognition to MCI in stroke-free individuals with AF. This study highlights the potential for targeted interventions aimed at slowing cognitive decline in stroke-free AF individuals by focusing on these structural alterations.