Frustrated magnetic interactions in FeSe

The structurally simplest high-temperature superconductor FeSe exhibits an intriguing superconducting nematic paramagnetic phase with unusual spin excitation spectra that are different from typical spin waves; thus, determining its effective magnetic exchange interactions is challenging. In this work, we report neutron scattering measurements of spin fluctuations of FeSe in the tetragonal paramagnetic phase. We show that the equal-time magnetic structure factor, S(Q), can be effectively modeled using the self-consistent Gaussian approximation calculation with highly frustrated nearest-neighbor (J₁) and next-nearest-neighbor (J₂) exchange couplings, and very weak further neighbor exchange interaction. Our results elucidate the frustrated magnetism in FeSe, which provides a natural explanation for the highly tunable superconductivity and nematicity in FeSe and related materials.