Deep-learning Autoencoder for Coherent and Nonlinear Optical Communication

Abstract: Motivated by the recent success of end-to-end training of communications in the wireless domain, we strive to adapt the end-to-end-learning idea from the wireless case (i.e., linear) to coherent optical fiber links (i.e., nonlinear). Although, at first glance, it sounds like a straightforward extension, it turns out that several pitfalls exist - in terms of theory but also in terms of practical implementation. This paper analyzes the potential of an autoencoder and limitations for the optical fiber under the influence of Kerr-nonlinearity and chromatic dispersion. As there is no exact capacity limit known and, hence, no analytical perfect system solution available, we set great value to the interpretability on the learnings of the autoencoder. Therefore, we design its architecture to be as close as possible to the structure of a classic communication system, knowing that this may limit its degree of freedom and, thus, its performance. Nevertheless, we were able to achieve an unexpected high gain in terms of spectral efficiency compared to a conventional reference system.