Analog Matrix Processor for Edge AI Real-Time Video Analytics
Laura Fick, Skylar Skrzyniarz, Malav P. Parikh, Michael Henry, David Fick
Abstract
One of the salient hurdles for wide adoption of machine learning (ML) has been efficient and high-performance edge compute. ML developers use very large, expensive, and power-intensive systems to create new models, but the final implementation needs to be deployed in a cost-effective, small, and low-power solution. This has commonly led to situations where the models that can be deployed in an edge environment are significantly smaller than state-of-the-art models leading the benchmark scoreboards [1]. A flash-based Analog Matrix Processor (AMP) is well positioned to close this development/deployment gap by providing edge inference compute that is 10-to-100× more efficient than traditional digital inference. It accomplishes this by leveraging denser circuitry via smaller memory elements (multi-level flash instead of several SRAM cells) and denser communication (256+ level signaling instead of binary signaling). Deep Neural Networks (DNNs) are an ideal application for the AMP since they can use analog-aware training to overcome the inherent non-idealities present in analog signal chains. DNNs use large pattern matchers which naturally suppress noise and map well to the AMP, which prefers to amortize the fixed costs inside the analog system over larger calculations.