Applications of Recurrent Neural Network

Abstract

Recurrent neural network (RNN) architecture is popular among researchers due to many of the advantages, among which most important are processing sequences of different lengths, handling vanishing gradient, and capability to capture temporal and spatial features in its hidden state within a sequence of inputs. RNNs have simple and hybrid architectures with a number of input-output combinations. In this chapter, the versatility of RNN is depicted through two case studies. Multiclass classification of EEG signal datasets is done using different RNN models to help diagnostic experts. The performance is analyzed with different numbers of layers and units of RNN architecture. The maximum efficiency of simple RNN, LSTM, and GRU is achieved with one layer and 1,024 units. With the increase in the number of layers with 32 units, the percentage increase in efficiency of LSTM along with time is better than that of simple RNN and GRU. Deep RNNs can process a complete sentence at a time instead of a word at a time and are thus suitable for signal analysis, sentiment analysis, handwriting and speech recognition, etc. The comparative evaluation of four deep learning–based models, namely decomposable attention, asymmetric attention, LSTM, and GRU, is carried out with respect to the performance, time for training, and number of trainable parameters. The models were tested on SNLI corpus, and it was found that the decomposable attention-based model achieved an accuracy of 86 percent. Hence decomposable attention model outperformed LSTM, GRU, and asymmetric attention model. It is observed that decomposable attention model has fewer trainable parameters and has significantly reduced training time compared to LSTM and GRU. The description of each case helps understand the significant role of RNN in practical examples.

Topics & Concepts