Metal–organic frameworks (MOFs) are often used as proton conductor fillers to enhance the proton conductivity of hybrid membranes. It is interesting but still challenging to discuss the influence of the molecular morphology and distribution of proton carriers within MOFs on their proton exchange membranes (PEMs) and their overall performance. In this work, two types of imidazole-loaded proton conductors were synthesized based on MIL-101(Cr), with one containing free imidazole molecules and the other containing imidazole molecules coordinating with Cr-metal sites. Furthermore, series composite membranes (CPM/IM-x and CPM@IM-x, x = 1, 2, 3) were fabricated using imidazole-loaded proton conductor as filler and a cross-linked poly(vinyl alcohol) (CL-PVA) that contains both hydrophilic and hydrophobic structures as matrix. The CPM/IM membranes with coordinated imidazole molecules exhibit significantly enhanced water affinity, stability, and ion exchange capacity. This is mainly due to the coordinated imidazole improving the two-phase dispersion, promoting imidazole deprotonation, and facilitating low-energy-barrier proton transport, while the CPM@IM membranes with the free imidazole molecules exhibit relatively lower proton conductivity but higher activation energy (Ea). Surprisingly, the proton conductivity of the CPM/IM-2 membrane can reach up to 2.7 × 10–2 S cm–1 at 85 °C and 90% RH, but with the lowest Ea of 20.9 kJ/mol. This remarkable performance indicates more efficient H+ transportation. Moreover, the single-cell power density of CPM/IM-2 is up to 89.7 mW cm–2, proving it exhibits high potential application prospects as a proton exchange membrane for hydrogen-based fuel cells.