Deep Neural Network Training Accelerator Designs in ASIC and FPGA
Shreyas Kolala Venkataramanaiah, Shihui Yin, Yu Cao, Jae-sun Seo
Abstract
In this invited paper, we present deep neural network (DNN) training accelerator designs in both ASIC and FPGA. The accelerators implements stochastic gradient descent based training algorithm in 16-bit fixed-point precision. A new cyclic weight storage and access scheme enables using the same off-the-shelf SRAMs for non-transpose and transpose operations during feed-forward and feed-backward phases, respectively, of the DNN training process. Including the cyclic weight scheme, the overall DNN training processor is implemented in both 65 nm CMOS ASIC and Intel Stratix-10 FPGA hardware. We collectively report the ASIC and FPGA training accelerator results.
Topics & Concepts
Application-specific integrated circuitTransposeField-programmable gate arrayComputer scienceStratixHardware accelerationArtificial neural networkComputer hardwareEmbedded systemArtificial intelligenceQuantum mechanicsPhysicsEigenvalues and eigenvectorsAdvanced Memory and Neural ComputingFerroelectric and Negative Capacitance DevicesAdvanced Neural Network Applications