Neutron diffraction was used to measure the total structure factors for several rare-earth ion ${R}^{3+}$ $({\mathrm{La}}^{3+}$ or ${\mathrm{Ce}}^{3+})$ phosphate glasses with composition close to $R{\mathrm{Al}}_{0.35}{\mathrm{P}}_{3.24}{\mathrm{O}}_{10.12}.$ By assuming isomorphic structures, difference function methods were employed to separate, essentially, those correlations involving ${R}^{3+}$ from the remainder. A self-consistent model of the glass structure was thereby developed in which the Al correlations were taken into explicit account. The glass network was found to be made from interlinked ${\mathrm{PO}}_{4}$ tetrahedra having 2.2(1) terminal oxygen atoms, ${\mathrm{O}}_{\mathrm{T}},$ at 1.51(1) \AA{}, and 1.8(1) bridging oxygen atoms, ${\mathrm{O}}_{\mathrm{B}},$ at 1.60(1) \AA{}. Rare-earth cations bonded to an average of 7.5(2) ${\mathrm{O}}_{\mathrm{T}}$ nearest neighbors in a broad and asymmetric distribution. The ${\mathrm{Al}}^{3+}$ ion acted as a network modifier and formed ${\mathrm{O}}_{\mathrm{T}}\ensuremath{-}\mathrm{Al}\ensuremath{-}{\mathrm{O}}_{\mathrm{T}}$ linkages that helped strengthen the glass. The connectivity of the R-centered coordination polyhedra was quantified in terms of a parameter ${f}_{s}$ and used to develop a model for the dependence on composition of the $\mathrm{Al}\ensuremath{-}{\mathrm{O}}_{\mathrm{T}}$ coordination number in $R\ensuremath{-}\mathrm{Al}\ensuremath{-}\mathrm{P}\ensuremath{-}\mathrm{O}$ glasses. By using recent ${}^{17}\mathrm{Al}$ nuclear-magnetic-resonance data, it was shown that this connectivity decreases monotonically with increasing Al content. The chemical durability of the glasses appeared to be at a maximum when the connectivity of the R-centered coordination polyhedra was at a minimum. The relation of ${f}_{s}$ to the glass transition temperature, ${T}_{g},$ was discussed.