# NuWa: Deriving Lightweight Class-Specific Vision Transformers for Edge Devices Source: [NuWa: Deriving Lightweight Class-Specific Vision Transformers for Edge Devices]() Collection index: [llms.txt]() Data updated: 2026-06-11T05:00:05.512Z Authors: Ziteng Wei, Qiang He, Bing Li, Feifei Chen, Hai Jin, Yun Yang Published: 2025-04-04T02:19:01.000Z ## Abstract Vision Transformers (ViTs) often need to be compressed for deployment on resource-constrained edge devices like drones and smart vehicles. However, existing model compression methods ignore that many edge devices only require the knowledge of specific classes for their applications. As a result, the derived all-class ViTs retain redundant knowledge and perform suboptimally on these classes. We discovered that simply replacing the calibration dataset with class-specific data does not suffice to address this issue, as these methods face two fundamental limitations. First, they overlook the existence of class-detrimental weights, which interfere with specialization, while removing them can improve class-specific performance. Second, the diversity of target classes and resource constraints on edge devices demand numerous customized models. Existing methods are time-consuming and computationally expensive, thus unscalable. In this work, we present NuWa, a cost-efficient method that addresses these challenges by deriving small ViTs from base ViTs for edge devices with specific class requirements. NuWa performs self-knowledge purification to prune class-detrimental weights and efficiently derives compact ViTs through closed-form optimization. Without post-pruning retraining, the derived edge ViTs surpass the base ViT in class-specific accuracy and accelerate inference. Comprehensive experiments demonstrate that NuWa outperforms state-of-the-art training-free pruning methods on class-specific tasks by up to 29.00\\% in accuracy. Compared with the best-performing training-dependent pruning method, NuWa achieves a 33.69x pruning speedup and reduces pruning cost by up to 99.83\\%, with only a 0.61\\% average accuracy loss. Project Page: https://github.com/CGCL-codes/NuWa. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on ImageNet sub-tasks S1-S3 (10 classes)](): Image Classification; ImageNet sub-tasks S1-S3 (10 classes); Accuracy (S1/10): 97. - [Image Classification on ImageNet sub-tasks S7-S9 (50 classes)](): Image Classification; ImageNet sub-tasks S7-S9 (50 classes); Accuracy (S7/50): 95.04. - [Image Classification on ImageNet sub-tasks S4-S6 (25 classes)](): Image Classification; ImageNet sub-tasks S4-S6 (25 classes); Accuracy (S4/25): 96.16. ## Original sources and implementation [Original paper]() [PDF]()