A Three-Wafer-Stacked Hybrid 15-MPixel CIS + 1-MPixel EVS With 4.6-GEvent/s Readout, In-Pixel TDC, and On-Chip ISP and ESP Function
Menghan Guo, Shoushun Chen, Zhe Gao, Wenlei Yang, Peter Bartkovjak, Qing Qin, Xiaoqin Hu, Dahai Zhou, Qiping Huang, Masayuki Uchiyama, Yoshiharu Kudo, Shimpei Fukuoka, Chengcheng Xu, Hiroaki Ebihara, Xueqing Wang, Peiwen Jiang, Bo Jiang, Bo Mu, Huan Chen, Jason Yang, Tian‐Jun Dai, Andreas Suess
Abstract
We introduce a hybrid 4096 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> 3680 CMOS image sensor (CIS) with an embedded 1032 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> 928 event-based vision sensor (EVS). Within a four-by-four cluster of CIS pixels, one color channel is substituted to provide a photocurrent for an EVS pixel. The EVS readout circuit is allocated on a second wafer and wrapped behind the four-by-four pixel cluster. A third wafer is used to integrate all peripheral readout circuitry for the EVS and CIS functions while achieving a compact sensor footprint. The EVS pixel achieves sub-1 Hz noise rates and 3% contrast non-uniformity (CTNU) at a linear nominal contrast threshold (NCT) of 15%. The row and column readout circuitry bypasses elements without events in a fraction of a clock cycle. Furthermore, a skip function is realized that reads only bordering pixels of a connected set of events. This achieves an event rate of up to 4.6 GEvents/s (GEps). The sensor captures in-pixel time stamps through sampling of a gray code timer realizing a time-to-digital conversion (TDC). Image and event signal processors (ISP & ESP) are integrated on-chip. The ESP provides global ambient and global activity monitoring functions. The sensor supports various flicker mitigation approaches such as adjustment of contrast threshold, in-pixel bandpass filter function and region-of-interest (ROI), and deterministic or random subsampling. The hybrid sensor mitigates challenges in synchronization and parallax error compared with a two-sensor solution using a dedicated CIS and dedicated EVS sensor. Furthermore, only one package and lens are required reducing footprint and cost.