Meta Learning for Natural Language Processing: A Survey

Hung-yi Lee, Shang-Wen Li, Thang X. Vu

2022Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies39 citationsDOI Open Access PDF

Abstract

Deep learning has been the mainstream technique in natural language processing (NLP) area. However, the techniques require many labeled data and are less generalizable across domains. Meta-learning is an arising field in machine learning studying approaches to learn better learning algorithms. Approaches aim at improving algorithms in various aspects, including data efficiency and generalizability. Efficacy of approaches has been shown in many NLP tasks, but there is no systematic survey of these approaches in NLP, which hinders more researchers from joining the field. Our goal with this survey paper is to offer researchers pointers to relevant meta-learning works in NLP and attract more attention from the NLP community to drive future innovation. This paper first introduces the general concepts of meta-learning and the common approaches. Then we summarize task construction settings and application of meta-learning for various NLP problems and review the development of meta-learning in NLP community.

Topics & Concepts

Artificial intelligenceComputer scienceMeta learning (computer science)Generalizability theoryMachine learningNatural language processingDeep learningAlgorithmic learning theoryInstance-based learningMulti-task learningField (mathematics)Task (project management)Active learning (machine learning)PsychologyEconomicsMathematicsDevelopmental psychologyPure mathematicsManagementTopic ModelingMachine Learning and Data ClassificationDomain Adaptation and Few-Shot Learning