Privacy in the Time of Language Models
Charith Peris, Christophe Dupuy, Jimit Majmudar, Rahil Parikh, Sami Smaili, Richard S. Zemel, Rahul Gupta
Abstract
Pretrained large language models (LLMs) have consistently shown state-of-the-art performance across multiple natural language processing (NLP) tasks. These models are of much interest for a variety of industrial applications that use NLP as a core component. However, LLMs have also been shown to memorize portions of their training data, which can contain private information. Therefore, when building and deploying LLMs, it is of value to apply privacy-preserving techniques that protect sensitive data.
Topics & Concepts
MemorizationComputer scienceVariety (cybernetics)Component (thermodynamics)Value (mathematics)Artificial intelligenceCore (optical fiber)Natural language processingMachine learningCognitive psychologyPsychologyPhysicsTelecommunicationsThermodynamicsPrivacy-Preserving Technologies in DataData Quality and ManagementCryptography and Data Security