# Locality Guidance for Improving Vision Transformers on Tiny Datasets

Source: [Locality Guidance for Improving Vision Transformers on Tiny Datasets](<https://www.sota2.com/research/paper/68fa6d191e209131cea37bb4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:25:36.384Z

Authors: Kehan Li, Runyi Yu, Zhennan Wang, Li Yuan, Guoli Song, Jie Chen

Published: 2022-07-20T16:41:41.000Z

## Abstract

While the Vision Transformer (VT) architecture is becoming trendy in computer vision, pure VT models perform poorly on tiny datasets. To address this issue, this paper proposes the locality guidance for improving the performance of VTs on tiny datasets. We first analyze that the local information, which is of great importance for understanding images, is hard to be learned with limited data due to the high flexibility and intrinsic globality of the self-attention mechanism in VTs. To facilitate local information, we realize the locality guidance for VTs by imitating the features of an already trained convolutional neural network (CNN), inspired by the built-in local-to-global hierarchy of CNN. Under our dual-task learning paradigm, the locality guidance provided by a lightweight CNN trained on low-resolution images is adequate to accelerate the convergence and improve the performance of VTs to a large extent. Therefore, our locality guidance approach is very simple and efficient, and can serve as a basic performance enhancement method for VTs on tiny datasets. Extensive experiments demonstrate that our method can significantly improve VTs when training from scratch on tiny datasets and is compatible with different kinds of VTs and datasets. For example, our proposed method can boost the performance of various VTs on tiny datasets (e.g., 13.07% for DeiT, 8.98% for T2T and 7.85% for PVT), and enhance even stronger baseline PVTv2 by 1.86% to 79.30%, showing the potential of VTs on tiny datasets. The code is available at https://github.com/lkhl/tiny-transformers.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on CIFAR-100](<https://www.sota2.com/research/sota/image-classification-on-cifar-100.md>): Image Classification; CIFAR-100; Accuracy: not recorded.

- [Medical Diagnosis Classification on Chaoyang](<https://www.sota2.com/research/sota/medical-diagnosis-classification-on-chaoyang.md>): Medical Diagnosis Classification; Chaoyang; F1 Score: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2207.10026>)

[PDF](<https://arxiv.org/pdf/2207.10026.pdf>)
