Innovative Optical Fiber Communication: New Benchmark for Linear Compensation Based on Learnable DSP

Figure 1 Optical Fiber Communication Technology

Furthermore, in practical applications, higher performance compensation often means greater complexity, which can lead to higher power consumption and larger device size, hindering system integration and promotion. In this context, how to improve DSP compensation performance while maintaining low complexity has become a significant challenge in the field of optical fiber communication research.

Learnable DSP Framework

To overcome the aforementioned difficulties, the research team proposed a novel DSP framework—Learnable DSP (LDSP). This scheme retains the traditional DSP framework while integrating a deep learning optimization framework. LDSP reuses traditional DSP modules, setting the parameters of each module to be learnable, and further optimizes them through deep learning algorithms to achieve global optimality. Specifically, LDSP processes in blocks, and during the processing of each signal block, each DSP module calculates gradients using the backpropagation algorithm and optimizes parameters with the stochastic gradient descent algorithm, as shown in Figure 2.

Technically, LDSP is a DSP framework with online training capabilities, combining the advantages of prior knowledge of optical communication and data-driven approaches. In this way, each compensation module in LDSP can share gradients to process information, comprehensively utilizing DSP resources for impairment compensation and enhancing transmission performance. In experiments involving 400Gb/s rate over 1600 kilometers of optical fiber transmission, LDSP improved Q-factor performance by approximately 0.77dB for single-channel and 0.56dB for 21-channel transmission. When combined with nonlinear compensation, performance improvements of up to 1.21dB and 0.9dB were achieved. This represents a new algorithm processing paradigm with the potential to revolutionize DSP solutions in optical fiber communication, and more importantly, it demonstrates that traditional linear DSP still has room for improvement, establishing a new benchmark for nonlinear compensation in optical fiber communication.

LDSP can implement multiple different functions within a single module, showcasing its efficiency. For example, the frequency domain compensation module can simultaneously compensate for IQ skew, laser frequency offset, and dispersion effects. Furthermore, by detecting sampling errors through the numerical value of IQ skew, it reduces the need for clock recovery modules, saving DSP resources. Additionally, the power normalization module can align the constellation diagram, and its bias can detect DAC clock leakage. The learned IQ skew and normalization bias are shown in Figure 3.

Figure 3 LDSP Learned Parameter Features

In traditional DSP systems, the parameters of each module are fixed and need to be manually adjusted according to actual conditions, which not only increases system complexity but may also lead to suboptimal compensation effects. In LDSP, these parameters are learnable and can be adaptively optimized through the backpropagation algorithm. LDSP can ensure high performance while enabling signal processing at symbol rates, reducing system complexity and making it more suitable for practical applications.