EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference
Thierry Tambe, Coleman Hooper, Lillian Pentecost, Tianyu Jia, En-Yu Yang, Marco Donato, Victor Sanh, Paul N. Whatmough, Alexander M. Rush, David Brooks, Gu-Yeon Wei
Abstract
Transformer-based language models such as BERT provide significant accuracy improvement to a multitude of natural language processing (NLP) tasks. However, their hefty computational and memory demands make them challenging to deploy to resource-constrained edge platforms with strict latency requirements.
Topics & Concepts
Computer scienceTransformerInferenceArtificial intelligenceNatural language processingLatency (audio)Language modelSentenceTask (project management)TelecommunicationsManagementVoltagePhysicsQuantum mechanicsEconomicsTopic ModelingNatural Language Processing TechniquesSpeech Recognition and Synthesis