Hidden representations in deep neural networks: Part 2. Regression problems
Laya Das, Abhishek Sivaram, Venkat Venkatasubramanian
Abstract
Deep neural networks are an important class of machine learning models useful for representing complex input-output relationships. While their recent success is unparalleled, so is the inability to explain their internal representations. In this second part of a two-part series, we focus on understanding the hidden representations of deep neural networks and the underlying mechanisms for regression problems. We highlight challenges associated with deep neural networks with simple models that help us gain insight into the functioning of the hidden layers and the mechanism of the operation of the network. The article is structured in a tutorial-like fashion for the benefit of new practitioners so that they can appreciate nuances and the pitfalls involved in developing a deep neural network models for regression problems.