Transverse mode instabilities in high power fiber amplifiers: a linear stability analysis

This work introduces a theoretical framework for analyzing transverse modal instability (TMI) in fiber amplifiers based on the stability of steady-state fundamental mode amplification under transverse perturbations. Unlike conventional stimulated thermal Rayleigh scattering (STRS) models, the approach directly incorporates the coupled effects of population inversion dynamics and thermally induced refractive index changes. The analysis identifies two distinct regimes: a population-inversion-dominated instability at low powers and small core diameters, and a thermally dominated regime at high powers, consistent with experimental observations. It further shows that TMI onset corresponds to a nonlinear phase-matching condition between the fundamental mode and transverse perturbations, arising from weak-wave retardation (cross-phase modulation). Compact analytical expressions for the TMI power threshold and characteristic frequency are derived, linking them explicitly to key fiber and amplifier parameters. The results provide a unified and physically intuitive description of TMI, offering new insight into power-scaling limits in high-power fiber systems.