Hardware-Efficient Duobinary Neural Network Equalizers for 800 Gb/s IM/DD PAM4 Transmission Over 10 km SSMF

• Published in: Journal of lightwave technology 2023-06, Vol.41 (12), p.3783-3790
• Main Authors: Bluemm, Christian, Liu, Bo, Li, Bing, Rahman, Talha, Hossain, Md Sabbir-Bin, Schaedler, Maximilian, Schlichtmann, Ulf, Kuschnerov, Maxim, Calabrò, Stefano
• Format: Article
• Language: English
• Subjects: Complexity; Data centers; Digital signal processing; Equalization; Hardware; Neural networks
• ISSN: 0733-8724; 1558-2213
• DOI: 10.1109/JLT.2023.3268579

In this article, we discuss challenges and options for scaling IM/DD transceivers towards 800 Gbps. Our focus is CWDM4 PAM4 transmission and our target distance is 10 km in O-band, which is a most urgent use case for next generation optical short reach systems like data centre interconnects and networks. At this reach and rate, chromatic dispersion (CD) becomes the main challenge. Its mitigation is essential and primarily done with digital signal processing. State of the art techniques, however, make transceivers quickly too complex. We show upon measurement results how neural network equalization can meet Volterra equalization performance with 30% less hardware multiplier complexity. When also applying magnitude weight pruning, an additional 43% reduction is possible without performance loss across all CWDM4 lanes. If needed, an added MLSE stage can further push performance in both cases. In any of these configurations, a key enabler against strong CD penalties is duobinary training, which is applicable to all feed-forward equalization architectures.