Surrogate-Assisted Cooperative Co-evolutionary Reservoir Architecture Search for Liquid State Machines
Yan Zhou, Yaochu Jin, Yao Hong Sun, Jinliang Ding
Abstract
Liquid state machines (LSMs) are biologically more plausible than feedforward spiking neural networks for brain-inspired computing and neuromorphic engineering. However, optimizing and training complex recurrent network architectures in the reservoir of LSMs remains challenging. Most existing algorithms aim to adjust the synaptic strength only, without fundamentally modifying the reservoir architecture of LSMs. Recently, it has become popular to simultaneously optimize the architecture and parameters of the reservoir in LSMs. However, most existing architecture representation schemes are too restricted to discover more powerful architectures of LSMs. To address the above issue, this paper proposes a generative liquid state machine, whose reservoir architecture is evolved using a cooperative co-evolutionary algorithm whose weights are tuned by synaptic plasticity rules. To reduce the computation time for evolving the reservoir, random forest is adopted to assist the cooperative co-evolutionary algorithm, together with a data parallelism strategy. The proposed algorithm is assessed on three sequence classification benchmarks and our experimental results show that the proposed algorithm outperforms the state-of-the-art on the benchmark problems. Meanwhile, our analysis shows that the data parallelism strategy is effective in speeding up the evaluation process.