In-sensor multilevel image adjustment for high-clarity contour extraction using adjustable synaptic phototransistors
Jong Ik Kwon, Ji Su Kim, Ji Su Kim, Hyojin Seung, Jihoon Kim, Jihoon Kim, Hanguk Cho, Tae-Min Choi, Jungwon Park, Juyoun Park, Jung Ah Lim, Moon Kee Choi, Dae‐Hyeong Kim, Changsoon Choi
Abstract
Robotic vision has traditionally relied on high-performance yet resource-intensive computing solutions, which necessitate high-throughput data transmission from vision sensors to remote computing servers, sacrificing energy efficiency and processing speed. A promising solution is data compaction through contour extraction, visualizing only the outlines of objects while eliminating superfluous backgrounds. Here, we introduce an in-sensor multilevel image adjustment method using adjustable synaptic phototransistors, enabling the capture of well-defined images with optimal brightness and contrast suitable for achieving high-clarity contour extraction. This is enabled by emulating dopamine-mediated neuronal excitability regulation mechanisms. Electrostatic gating effect either facilitates or inhibits time-dependent photocurrent accumulation, adjusting photo-responses to varying lighting conditions. Through excitatory and inhibitory modes, the adjustable synaptic phototransistor enhances visibility of dim and bright regions, respectively, facilitating distinct contour extraction and high-accuracy semantic segmentation. Evaluations using road images demonstrate improvement of both object detection accuracy and intersection over union, and compression of data volume.