Ultralow Complexity Long Short-Term Memory Network for Fiber Nonlinearity Mitigation in Coherent Optical Communication Systems

Fiber Kerr nonlinearity is a fundamental limitation to the achievable capacity of long-distance optical fiber communication. Digital back-propagation (DBP) is a primary methodology to mitigate both linear and nonlinear impairments by solving the inverse-propagating nonlinear Schrödinger equation (NLSE), which requires detailed link information. Recently, the paradigms based on neural network (NN) were proposed to mitigate nonlinear transmission impairments in optical communication systems. However, almost all neural network-based equalization schemes yield high computation complexity, which prevents the practical implementation in commercial transmission systems. In this paper, we propose a center-oriented long short-term memory network (Co-LSTM) incorporating a simplified mode with a recycling mechanism in the equalization operation, which can mitigate fiber nonlinearity in coherent optical communication systems with ultralow complexity. To validate the proposed methodology, we carry out an experiment of ten-channel wavelength division multiplexing (WDM) transmission over 1600 km standard single-mode fiber (SSMF) with 64 Gbaud polarization-division-multiplexed 16-ary quadrature amplitude modulation (16-QAM) signals. A 0.51 dB Q² factor gain is observed with the Co-LSTM equalization, which is comparable to that of DBP. The complexity of the Co-LSTM equalization is only 5.2% of that of the conventional bi-directional LSTM, and 28.4% of that of the DBP method with a single step per span. In principle, the complexity of the Co-LSTM with a simplified mode is almost independent of the transmission distance, which shows an essential benefit over the DBP method that determined by the optical signal evolution along the fiber link. The proposed Co-LSTM methodology presents an attractive approach for low complexity nonlinearity mitigation with neural networks.