Sample-Based Neural Network Pre-Distorter for Transceiver Nonlinearity Compensation in IM/DD System

Look-up table (LUT) enabled digital pre-distortion (DPD) is an effective means of nonlinear compensation. However, considering the storage requirement and feasibility of training process, the memory length is commonly limited to 3 or 5, which limits the nonlinear compensation performance of LUT-DPD. With the growing demands of high baud rate transmission, the system is more sensitive to nonlinear impairments, and LUT-DPD with short memory length is insufficient to meet the requirement on nonlinear compensation performance. Neural network (NN) has been proposed as a pre-distorter, while it requires precise modeling of the transceiver or complex multiplication operation. Following the training process of LUT, NN-based pre-distorter can also be trained on received samples, which is simpler to be implemented. Besides, giving scope to the learning ability of NN, a small number of samples are enough for training process. Moreover, the memory length can be easily expanded in NN-based method. Therefore, in this paper, we propose a sample-based NN-DPD which extends the memory length to 9 and even 13, with relatively low complexity. We experimentally demonstrate a transmission of beyond 100 Gbit/s PAM-6/PAM-8 signal in an intensity modulation and direct detection (IM/DD) system. For 40 Gbaud PAM-6 signal, a maximum receiver sensitivity gain of 1.5 dB is obtained at the KP4-FEC threshold when memory length is increased from 3 to 13. Compared to Volterra nonlinear equalizer (VNLE), the computational complexity of NN-DPD is reduced by nearly 60% when memory length is 9. The NN-DPD is a promising solution to reducing the influence of nonlinearity for future high-order modulation signal.