# HSViT: Horizontally Scalable Vision Transformer Source: [HSViT: Horizontally Scalable Vision Transformer]() Collection index: [llms.txt]() Data updated: 2025-10-26T03:03:52.408Z Authors: Chenhao Xu, Chang-Tsun Li, Chee Peng Lim, Douglas Creighton Published: 2024-04-08T04:53:29.000Z ## Abstract Due to its deficiency in prior knowledge (inductive bias), Vision Transformer (ViT) requires pre-training on large-scale datasets to perform well. Moreover, the growing layers and parameters in ViT models impede their applicability to devices with limited computing resources. To mitigate the aforementioned challenges, this paper introduces a novel horizontally scalable vision transformer (HSViT) scheme. Specifically, a novel image-level feature embedding is introduced to ViT, where the preserved inductive bias allows the model to eliminate the need for pre-training while outperforming on small datasets. Besides, a novel horizontally scalable architecture is designed, facilitating collaborative model training and inference across multiple computing devices. The experimental results depict that, without pre-training, HSViT achieves up to 10% higher top-1 accuracy than state-of-the-art schemes on small datasets, while providing existing CNN backbones up to 3.1% improvement in top-1 accuracy on ImageNet. The code is available at https://github.com/xuchenhao001/HSViT. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on TinyImageNet (test)](): Image Classification; TinyImageNet (test); Accuracy: 56.73. ## Original sources and implementation [Original paper]() [PDF]()