Interlayer exchange tuned magnetotransport properties in the kagome antiferromagnet YMn6Sn6

Kagome magnets have attracted significant attention due to the interplay between magnetic orders and topological electronic states. The kagome compound ${\mathrm{YMn}}_{6}{\mathrm{Sn}}_{6}$ exhibits a helical antiferromagnetic ground state due to the frustrated interlayer exchange interactions, and undergoes successive in-plane field-induced magnetic transitions, resulting in complex magnetotransport properties. In this work, we investigate the magnetic and magnetotransport properties of Tb-substituted ${\mathrm{YMn}}_{6}{\mathrm{Sn}}_{6}$ single crystals. Through low-level doping, the interlayer antiferromagnetic exchange is reduced without altering the magnetic structure. Compared to the pristine sample, the doped samples exhibit larger negative magnetoresistance, sharper field-induced resistance transitions, and smaller topological Hall effect. These behaviors can be attributed to the reduction of magnetic fluctuations, which is caused by the shifted balance between the ferromagnetic and antiferromagnetic exchanges. In addition, we analyze the anomalous negative contribution in low-temperature Hall resistivity, which occurs within the same magnetic field range as the negative magnetoresistance in the fanlike magnetic phase. We propose that this phenomenon arises from the magnetization-driven Lifshitz transition.