Programmable Tanh- and ELU-Based Photonic Neurons in Optics-Informed Neural Networks
Stefanos Kovaios, Christos Pappas, Miltiadis Moralis‐Pegios, Apostolos Tsakyridis, George Giamougiannis, Manos Kirtas, Joris Van Kerrebrouck, Gertjan Coudyzer, Xin Yin, Nikolaos Passalis, Anastasios Tefas, Nikos Pleros
Abstract
We demonstrate an integrated opto-electronic (ΟΕ) device that can be programmed to provide a set of nonlinear activation functions (AFs) and present its operation within programmable tanh- and ELU-based photonic neurons at line rates up to 10 GBd. The OE activation module provides a set of well-known activation functions that are typically used in DL training models, including the tanh-, ELU- and inverted ELU-like functions. Its performance is experimentally evaluated when incorporated in a 4-input wavelength division multiplexed (WDM) photonic neuron and operating with non-deterministic data patterns, providing “noisy” tanh, ELU and inverted ELU AFs with an error-distribution that has in all cases a standard deviation of <0.49. We also evaluate the trainability of these “noisy” AFs and present for the first time an optics-informed training framework that incorporates the pattern-induced AF variations into the training process, yielding the first noise-aware training scheme where the noise emerges at the nonlinear AF NN segment. The performance analysis of the optics-informed training framework for all three AFs was carried out via Deep Learning setups suitable for classifying the Fashion MNIST and the CIFAR-10 datasets. This analysis has shown that the employment of traditional training schemes leads to significant accuracy degradations, which can be, however, almost completely waived when employing the optics-informed training framework, leading to accuracy values that are almost identical to the reference accuracy values obtained when ideal and noise-less AFs are used.