A Universal Representation Transformer Layer for Few-Shot Image Classification
Lu Liu, William L. Hamilton, Guodong Long, Jing Jiang, Hugo Larochelle
Abstract
Few-shot classification aims to recognize unseen classes when presented with\nonly a small number of samples. We consider the problem of multi-domain\nfew-shot image classification, where unseen classes and examples come from\ndiverse data sources. This problem has seen growing interest and has inspired\nthe development of benchmarks such as Meta-Dataset. A key challenge in this\nmulti-domain setting is to effectively integrate the feature representations\nfrom the diverse set of training domains. Here, we propose a Universal\nRepresentation Transformer (URT) layer, that meta-learns to leverage universal\nfeatures for few-shot classification by dynamically re-weighting and composing\nthe most appropriate domain-specific representations. In experiments, we show\nthat URT sets a new state-of-the-art result on Meta-Dataset. Specifically, it\nachieves top-performance on the highest number of data sources compared to\ncompeting methods. We analyze variants of URT and present a visualization of\nthe attention score heatmaps that sheds light on how the model performs\ncross-domain generalization. Our code is available at\nhttps://github.com/liulu112601/URT.