Optimal Input Representation in Neural Systems at the Edge of Chaos
Guillermo B. Morales, Miguel A. Muñoz
Abstract
Shedding light on how biological systems represent, process and store information in noisy \nenvironments is a key and challenging goal. A stimulating, though controversial, hypothesis poses \nthat operating in dynamical regimes near the edge of a phase transition, i.e., at criticality or the “edge \nof chaos”, can provide information-processing living systems with important operational advantages, \ncreating, e.g., an optimal trade-off between robustness and flexibility. Here, we elaborate on a recent \ntheoretical result, which establishes that the spectrum of covariance matrices of neural networks \nrepresenting complex inputs in a robust way needs to decay as a power-law of the rank, with an \nexponent close to unity, a result that has been indeed experimentally verified in neurons of the mouse \nvisual cortex. Aimed at understanding and mimicking these results, we construct an artificial neural \nnetwork and train it to classify images. We find that the best performance in such a task is obtained \nwhen the network operates near the critical point, at which the eigenspectrum of the covariance \nmatrix follows the very same statistics as actual neurons do. Thus, we conclude that operating near \ncriticality can also have—besides the usually alleged virtues—the advantage of allowing for flexible, \nrobust and efficient input representations.