Machine learning based mitigation of fiber nonlinearity caused by chaotic signals in optical communication

Abstract High-capacity and long-distance optical fiber communication systems are increasingly challenged by non-linear factors that impair signal quality. Chaotic signals cause excessive fiber nonlinearity, which distorts signal transmission and lowers system performance. Traditional mitigation solutions, such as hardware correction or signal processing, fail under rapidly changing signal dynamics. This unique approach, the enhanced residual deep denoising network (ERDDNet), reduces fiber non-linearity caused by chaotic signals in optical networks. The ERDDNet method dynamically detects and suppresses nonlinear distortions without requiring knowledge of the channel model. The method is trained on a synthetic dataset of chaotic signal patterns and fiber-distorted outputs. It learns sophisticated non-linear optical transmission relationships this way. Experimental simulations show that the proposed ERDDNet technique reduces bit error rate by up to 38% compared to digital backpropagation. The program also adjusts to different fiber lengths and signal intensities, proving its durability in dynamic situations. This improves transmission reliability and provides new paths for AI-driven signal processing in next-generation optical communications.