Skip to main navigation Skip to search Skip to main content

CapViT: Cross-context capsule vision transformers for land cover classification with airborne multispectral LiDAR data

  • Yongtao Yu*
  • , Tao Jiang
  • , Junyong Gao
  • , Haiyan Guan
  • , Dilong Li
  • , Shangbing Gao
  • , E. Tang
  • , Wenhao Wang
  • , Peng Tang
  • , Jonathan Li
  • *Corresponding author for this work
  • Huaiyin Institute of Technology
  • Nanjing University of Information Science & Technology
  • Huaqiao University
  • University of Waterloo

Research output: Contribution to journal › Review article › peer-review

Abstract

Equipped with multiple channels of laser scanners, multispectral light detection and ranging (MS-LiDAR) devices possess more advanced prospects in earth observation tasks compared with their single-band counterparts. It also opens up a potential-competitive solution to conducting land cover mapping with MS-LiDAR devices. In this paper, we develop a cross-context capsule vision transformer (CapViT) to serve for land cover classification with MS-LiDAR data. Specifically, the CapViT is structurized with three streams of capsule transformer encoders, which are stacked by capsule transformer (CapFormer) blocks, to exploit long-range global feature interactions at different context scales. These cross-context feature semantics are finally effectively fused to supervise accurate land cover type inferences. In addition, the CapFormer block parallels dual-path multi-head self-attention modules functioning to interpret both spatial token correlations and channel feature interdependencies, which favor significantly to the semantic promotion of feature encodings. Consequently, with the semantic-promoted feature encodings to boost the feature representation distinctiveness and quality, the land cover classification accuracy is effectively improved. The CapViT is elaborately testified on two MS-LiDAR datasets. Both quantitative assessments and comparative analyses demonstrate the competitive capability and advanced performance of the CapViT in tackling land cover classification issues.

Original languageEnglish
Article number102837
JournalInternational Journal of Applied Earth Observation and Geoinformation
Volume111
DOIs
StatePublished - Jul 2022
Externally publishedYes

Keywords

  • Capsule network
  • Cross-context feature
  • Feature self-attention
  • Land cover
  • Multispectral light detection and ranging (MS-LiDAR)
  • Vision transformer

Fingerprint

Dive into the research topics of 'CapViT: Cross-context capsule vision transformers for land cover classification with airborne multispectral LiDAR data'. Together they form a unique fingerprint.

Cite this