Dynamic self-ensemble on multi-exit residual vision transformer for small-data image classification

Key points

• Vision Transformer (ViT) has garnered increased attention for image classification in recent years.1
• In this paper, we study how ViT specifically works on small-scale datasets with limited data where the availability of image data is often constrained in practice, especially in fields such as medical imaging.1
• To mitigate the issue, we designed a multi-stage residual ViT, dubbed ReSEViT, with the self-ensemble method to utilize multi-level information from intermediate features.1
• The integration of residual tokens and self-ensemble learning into a multi-stage ViT highlights the reduction in redundancy, prediction error and variance while maintaining an efficient model size for small-scale image datasets.1

Sources (1)

• [1]Dynamic self-ensemble on multi-exit residual vision transformer for small-data image classification
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 10:07 AM

  Hence, we propose a new learning method for ViT, named self-ensemble learning, which ensembles multi-stage classifiers in a single model.

  Vision Transformer (ViT) has garnered increased attention for image classification in recent years.

Extractive summary: sentences quoted from the sources.