Kan: Knowledge-augmented networks for few-shot learning

Zeyang Zhu, Xin Lin

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

Few-shot learning task aims to explore a model that is able to quickly learn new concepts by learning a few examples. The current approaches learning new categories with few images or even a single image are only based on the the visual modality. However, it is difficult to learn the representative features of new categories by a few images. This is because some categories are similar in vision. Moreover, due to the viewpoint, luminosity and that sometimes individuals of the same species appear markedly different from one another, the models are not able to learn the exact representation of classes. Therefore, considering that semantic information can enhance understanding when visual information is limited, we propose Knowledge-Augmented Networks (KAN), which combines the visual features with the semantic information extracted from knowledge graph to represent the features of each class. We demonstrate the effectiveness of our method on standard few-shot learning tasks, and further observe that with the augmented semantic information from knowledge graph, KAN is able to learn more disentangled representations. Experiments show that our model outperforms the state-of-the-art methods.

Original languageEnglish
Title of host publication2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1735-1739
Number of pages5
ISBN (Electronic)9781728176055
DOIs
StatePublished - 2021
Event2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021 - Virtual, Toronto, Canada
Duration: 6 Jun 202111 Jun 2021

Publication series

NameICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume2021-June
ISSN (Print)1520-6149

Conference

Conference2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021
Country/TerritoryCanada
CityVirtual, Toronto
Period6/06/2111/06/21

Keywords

  • Few-Shot Learning
  • Image Classification
  • Multimodal Fusion

Fingerprint

Dive into the research topics of 'Kan: Knowledge-augmented networks for few-shot learning'. Together they form a unique fingerprint.

Cite this