CogTaskonomy: Cognitively Inspired Task Taxonomy Is Beneficial to Transfer Learning in NLP
Yifei Luo, Minghui Xu, Deyi Xiong
Abstract
Is there a principle to guide transfer learning across tasks in natural language processing (NLP)? Taxonomy In this paper, we propose a cognitively inspired framework, CogTaskonomy, to learn taxonomy for NLP tasks. The framework consists of Cognitive Representation Analytics (CRA) and Cognitive-Neural Mapping (CNM). The former employs Representational Similarity Analysis, which is commonly used in computational neuroscience to find a correlation between brainactivity measurement and computational modeling, to estimate task similarity with taskspecific sentence representations. The latter learns to detect task relations by projecting neural representations from NLP models to cognitive signals (i.e., fMRI voxels). Experiments on 12 NLP tasks, where BERT/TinyBERT are used as the underlying models for transfer learning, demonstrate that the proposed Cog-Taskonomy is able to guide transfer learning, achieving performance competitive to the Analytic Hierarchy Process Analyses further discover that CNM is capable of learning modelagnostic task taxonomy. The source code is available at https://github.com/ tjunlp-lab/CogTaskonomy.git.