# A Unified Pruning Framework for Vision Transformers

Source: [A Unified Pruning Framework for Vision Transformers](<https://www.sota2.com/research/paper/68fa6b4d1e209131cea2d705>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:45:08.233Z

Authors: Hao Yu, Jianxin Wu

Published: 2021-11-30T05:01:02.000Z

## Abstract

Recently, vision transformer (ViT) and its variants have achieved promising performances in various computer vision tasks. Yet the high computational costs and training data requirements of ViTs limit their application in resource-constrained settings. Model compression is an effective method to speed up deep learning models, but the research of compressing ViTs has been less explored. Many previous works concentrate on reducing the number of tokens. However, this line of attack breaks down the spatial structure of ViTs and is hard to be generalized into downstream tasks. In this paper, we design a unified framework for structural pruning of both ViTs and its variants, namely UP-ViTs. Our method focuses on pruning all ViTs components while maintaining the consistency of the model structure. Abundant experimental results show that our method can achieve high accuracy on compressed ViTs and variants, e.g., UP-DeiT-T achieves 75.79% accuracy on ImageNet, which outperforms the vanilla DeiT-T by 3.59% with the same computational cost. UP-PVTv2-B0 improves the accuracy of PVTv2-B0 by 4.83% for ImageNet classification. Meanwhile, UP-ViTs maintains the consistency of the token representation and gains consistent improvements on object detection tasks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on ImageNet-1K (val) (Top-1 and Top-5 Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-val-top-1-and-top-5-accuracy.md>): Image Classification; ImageNet-1K 1 (val); Top-1 Accuracy: 75.79.

- [Image Classification on CIFAR-100 (val)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-val.md>): Image Classification; CIFAR-100 (val); Accuracy: not recorded.

- [Image Classification on TinyImageNet (val)](<https://www.sota2.com/research/sota/image-classification-on-tinyimagenet-val.md>): Image Classification; TinyImageNet (val); Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2111.15127>)

[PDF](<https://arxiv.org/pdf/2111.15127.pdf>)