Online learning using deep random vector functional link network
Sreenivasan Shiva, Minghui Hu, Ponnuthurai Nagaratnam Suganthan
Abstract
Deep neural networks have shown their promise in recent years with their state-of-the-art results. Yet, backpropagation-based methods may suffer from time-consuming training process and catastrophic forgetting when performing online learning. In this work we attempt to curtail them by employing the ensemble deep Random Vector Functional Link (edRVFL). As opposed to backpropagation-based neural networks that adjust weights iteratively, RVFL uses a closed-form solution method without iterative parameter learning. In addition, our approach allows the model to grow incrementally as new data is made available so that it can more resemble real-life learning scenarios. Our proposed online learning models were able to perform better on 72% of the datasets in the classification scenario and 80% of the datasets in the regression scenario, when compared to other available randomization-based online learning models in the literature. This is further supported by statistical comparisons which also show the stability of our network.