An Interpretable Constructive Algorithm for Incremental Random Weight Neural Networks and Its Application

Abstract

In this article, we aim to offer an interpretable learning paradigm for incremental random weight neural networks (IRWNNs). IRWNNs have become a hot research direction of neural network algorithms due to their ease of deployment and fast learning speed. However, existing IRWNNs have difficulty explaining how hidden nodes (parameters) affect the convergence of network residuals. To address this gap, this article proposes an interpretable construction algorithm (ICA). Specifically, we first conduct a spatial geometric analysis of the network construction process and establish the spatial geometric relationship between the network residuals and hidden parameters to visualize the influence of hidden parameters on the convergence of the network residuals. Second, based on the spatial geometric relationship and node pool strategy, an interpretable control strategy with spatial geometry information is established to obtain hidden parameters conducive to the convergence of network residuals. In addition, to facilitate ICA to handle complex tasks of big data, this article proposes a lightweight ICA with low complexity, namely ICA+. Finally, it is proved theoretically that the ICA and ICA+ proposed in this article have universal approximation properties. The experimental results on two real-world datasets and seven benchmark datasets demonstrate the advantages of the proposed ICA and ICA+ in terms of fast learning, good generalization, and compactness of network structure.

Topics & Concepts