Analytical Guarantees for Hyperparameter Free RFF Based Deep Learning in the Low-Data Regime
Rangeet Mitra, Georges Kaddoum, Gwenael Poitau
Abstract
Deep-learning (DL) has emerged as promising for diverse signal-processing tasks concerning the next-generation of communication-systems, Internet of Things (IoT), and in circuit-design. However, existing DL based signal-processing methods are well-known to require large datasets to achieve good generalization. Moreover, the existing DL-based solutions are known for their sensitivity to the hyperparameter-choices and the DL-architecture. In this regard, existing DL-paradigms, such as meta-learning are useful to find optimal hyperparameter-values and architectures. In this work, generalized analytical results are provided for random Fourier feature-based DL (RFF-DL) that guarantee a degree of hyperparameter-independence and significantly improved generalization under data-limited applications. The benefits of RFF-DL promised by the presented analytical contributions are validated by a case study that improves low-pilot training for the next-generation IoT using the hyperparameter-free RFF based DL.