Bioinspired high-order in-sensor spatiotemporal enhancement in van der Waals optoelectronic neuromorphic electronics
Mengjiao Li, Hongling Chu, Caifang Gao, Feng‐Shou Yang, Muyun Huang, L. Miu, Jun Li, Ching‐Hwa Ho, Jingjing Liu, Yen‐Fu Lin, Jianhua Zhang
Abstract
In over-complicated machine vision, target tracking within deep learning paradigms yields inaccurate and energy-intensive outputs. Although spiking neural networks excel at processing dynamic information, challenging tracking environments demand further enhancement in feature correlation learning for efficient target tracking. Distinct from Paired-spike-timing-dependent-plasticity-based architectures, we demonstrate a visual sensor based on van der Waals phototransistors, leveraging Triplet-spike-timing-dependent plasticity to extract bioinspired high-order correlation information, through tunable light-electric cooperation and competition effect on synaptic plasticity originating from interfacial defects-dominated persistent photoconductance phenomena. The universal Triplet-spike-timing-dependent plasticity with enhanced spatiotemporal correlation learning characteristic renders spiking neural networks with better processing capabilities for confusing object classification and dynamic tracking (90.44%) tasks, excelling particularly in seamless tracking post-occlusion, furthermore experimentally validated through hardware implementation on a 6 $$\times$$ 6 van der Waals phototransistor array. The offers a bottom-up methodology employing device physics to guide mapping of biorational learning for high-performance dynamic tracking towards advanced machine visual technologies. Li et al. report a high-order spatiotemporal processing vision sensor based on vdW phototransistors, leveraging Triplet-spike-timing dependent plasticity to extract the high-order correlation information, rendering spiking neural network for confusing object classifying and dynamic tracking.