Image Classification on ImageNet-100 (val)
89.11Top-1 Accuracyasymmetric knowledge distillation framework
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| asymmetric knowledge distillation frameworkDistortion=Gaussian Blur n = 212026.04 | 89.11 | — | — | — | — | — | — | — | — | |
| asymmetric knowledge distillation frameworkDistortion=Gaussian Noise σ = 0.32026.04 | 88.77 | — | — | — | — | — | — | — | — | |
| EQ-VMamba-SArch.=SSM, #Param. (M)=17M2026.03 | 88.7 | 98.22 | — | — | — | — | — | — | — | |
| EQ-VMamba-TArch.=SSM, #Param. (M)=10M2026.03 | 88.58 | 98.14 | — | — | — | — | — | — | — | |
| MSVMamba-TArch.=SSM, #Param. (M)=33M2026.03 | 88.44 | 97.92 | — | — | — | — | — | — | — | |
| VMamba-SArch.=SSM, #Param. (M)=50M2026.03 | 88.32 | 97.82 | — | — | — | — | — | — | — | |
| Clean imagesBackbone=frozen CLIP ViT-L/142026.06 | 88.26 | — | — | — | — | — | — | — | — | |
| SpectralVMamba-SArch.=SSM, #Param. (M)=35M2026.03 | 88.09 | 97.08 | — | — | — | — | — | — | — | |
| SpectralVMamba-TArch.=SSM, #Param. (M)=21M2026.03 | 87.86 | 97.25 | — | — | — | — | — | — | — | |
| VMamba-TArch.=SSM, #Param. (M)=30M2026.03 | 87.8 | 97.7 | — | — | — | — | — | — | — | |
| XCiT-M24Arch.=Trans., #Param. (M)=84M2026.03 | 87.72 | 97.36 | — | — | — | — | — | — | — | |
| MSVMamba-MArch.=SSM, #Param. (M)=12M2026.03 | 87.56 | 97.76 | — | — | — | — | — | — | — | |
| ConvNeXt-SArch.=CNN, #Param. (M)=50M2026.03 | 87.54 | 96.96 | — | — | — | — | — | — | — | |
| Supervised BaselineDistortion=Gaussian Noise σ = 0.32026.04 | 87.48 | — | — | — | — | — | — | — | — | |
| Supervised BaselineDistortion=Gaussian Blur n = 212026.04 | 87.46 | — | — | — | — | — | — | — | — | |
| Swin-TArch.=Trans., #Param. (M)=29M2026.03 | 87.42 | 97.2 | — | — | — | — | — | — | — | |
| Swin-SArch.=Trans., #Param. (M)=50M2026.03 | 87.26 | 97.46 | — | — | — | — | — | — | — | |
| XCiT-S24Arch.=Trans., #Param. (M)=26M2026.03 | 87.14 | 96.98 | — | — | — | — | — | — | — | |
| ConvNeXt-TArch.=CNN, #Param. (M)=29M2026.03 | 87.06 | 96.7 | — | — | — | — | — | — | — | |
| FPBackbone=ResNet-342025.03 | 86.87 | — | — | — | — | — | — | — | — | |
| FPBackbone=ResNet-182025.03 | 86.77 | — | — | — | — | — | — | — | — | |
| HOTBackbone=ResNet-342025.03 | 86.7 | — | — | — | — | — | — | — | — | |
| CIDistortion=Random Mask 75%2026.04 | 86.27 | — | — | — | — | — | — | — | — | |
| HOTBackbone=ResNet-182025.03 | 86.26 | — | — | — | — | — | — | — | — | |
| Robust EncoderDistortion=Random Mask 50%, Labeled Data fraction=100%, Evaluation protocol=Linear probe2021.10 | 85.87 | — | — | — | — | — | — | — | — | |
| FPBackbone=ResNet-502025.03 | 85.51 | — | — | — | — | — | — | — | — | |
| asymmetric knowledge distillation frameworkDistortion=Random Mask 75%2026.04 | 85.41 | — | — | — | — | — | — | — | — | |
| randomBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.252026.02 | 85.3 | — | — | — | — | — | — | — | — | |
| HOTBackbone=ResNet-502025.03 | 85.2 | — | — | — | — | — | — | — | — | |
| CMR-LogitBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.752026.02 | 85.16 | — | — | — | — | — | — | — | — | |
| CMR-AffineBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.252026.02 | 85.16 | — | — | — | — | — | — | — | — | |
| CMR-AffineBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.502026.02 | 85.04 | — | — | — | — | — | — | — | — | |
| VBPBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.502026.02 | 85 | — | — | — | — | — | — | — | — | |
| CMR-ConstBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.252026.02 | 84.96 | — | — | — | — | — | — | — | — | |
| CMR-LogitBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.252026.02 | 84.88 | — | — | — | — | — | — | — | — | |
| randomBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.502026.02 | 84.82 | — | — | — | — | — | — | — | — | |
| CMR-LogitBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.502026.02 | 84.78 | — | — | — | — | — | — | — | — | |
| magnitudeBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.502026.02 | 84.78 | — | — | — | — | — | — | — | — | |
| CIDistortion=Gaussian Blur n = 212026.04 | 84.72 | — | — | — | — | — | — | — | — | |
| CMR-AffineBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.752026.02 | 84.7 | — | — | — | — | — | — | — | — | |
| VBPBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.252026.02 | 84.7 | — | — | — | — | — | — | — | — | |
| magnitudeBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.752026.02 | 84.7 | — | — | — | — | — | — | — | — | |
| randomBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.752026.02 | 84.68 | — | — | — | — | — | — | — | — | |
| CMR-ConstBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.752026.02 | 84.66 | — | — | — | — | — | — | — | — | |
| CMR-ConstBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.502026.02 | 84.66 | — | — | — | — | — | — | — | — | |
| asymmetric knowledge distillation frameworkDistortion=Gaussian Noise σ = 0.52026.04 | 84.64 | — | — | — | — | — | — | — | — | |
| magnitudeBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.252026.02 | 84.62 | — | — | — | — | — | — | — | — | |
| VBPBackbone=DeiT-Tiny, Fine-tuning=matched fine-tuning, Retained fraction (rho)=0.752026.02 | 84.5 | — | — | — | — | — | — | — | — | |
| Robust EncoderDistortion=Gaussian Noise sigma = 0.1, Labeled Data fraction=100%, Evaluation protocol=Linear probe2021.10 | 84.46 | — | — | — | — | — | — | — | — | |
| Baseline (Original Model)Backbone=ViT-B/162026.04 | 84.34 | — | — | — | — | — | — | — | — | |
| Robust EncoderDistortion=Random Mask 75%, Labeled Data fraction=100%, Evaluation protocol=Linear probe2021.10 | 83.99 | — | — | — | — | — | — | — | — | |
| asymmetric knowledge distillation frameworkDistortion=Gaussian Blur n = 372026.04 | 83.94 | — | — | — | — | — | — | — | — | |
| CIDistortion=Random Mask 90%2026.04 | 83.92 | — | — | — | — | — | — | — | — | |
| FPBackbone=EfficientFormer-L12025.03 | 83.38 | — | — | — | — | — | — | — | — | |
| LBP-WHTBackbone=ResNet-342025.03 | 83.31 | — | — | — | — | — | — | — | — | |
| FPBackbone=EfficientFormer-L32025.03 | 83.3 | — | — | — | — | — | — | — | — | |
| Robust EncoderDistortion=Gaussian Blur n = 21, Labeled Data fraction=100%, Evaluation protocol=Linear probe2021.10 | 83.24 | — | — | — | — | — | — | — | — | |
| LUQBackbone=EfficientFormer-L32025.03 | 83.13 | — | — | — | — | — | — | — | — | |
| JumpReLUBackbone=ViT-B/16, Tokens=All, Layer Range=All2026.04 | 83.12 | — | — | — | — | — | — | — | — | |
| HOTBackbone=EfficientFormer-L12025.03 | 83.05 | — | — | — | — | — | — | — | — | |
| CIDistortion=Gaussian Noise σ = 0.32026.04 | 83.04 | — | — | — | — | — | — | — | — | |
| HOTBackbone=EfficientFormer-L32025.03 | 83.01 | — | — | — | — | — | — | — | — | |
| Robust EncoderDistortion=Random Mask 90%, Labeled Data fraction=100%, Evaluation protocol=Linear probe2021.10 | 82.96 | — | — | — | — | — | — | — | — | |
| Supervised BaselineDistortion=Random Mask 75%2026.04 | 82.9 | — | — | — | — | — | — | — | — | |
| LUQBackbone=ResNet-182025.03 | 82.77 | — | — | — | — | — | — | — | — | |
| CorInfoMaxBackbone=ResNet-50, Evaluation Protocol=Linear Evaluation2022.09 | 82.64 | — | — | — | — | — | — | — | — | |
| LUQBackbone=ResNet-342025.03 | 82.6 | — | — | — | — | — | — | — | — | |
| Supervised BaselineDistortion=Gaussian Noise σ = 0.52026.04 | 82.56 | — | — | — | — | — | — | — | — | |
| LUQBackbone=EfficientFormer-L12025.03 | 82.46 | — | — | — | — | — | — | — | — | |
| LBP-WHTBackbone=ResNet-182025.03 | 82.31 | — | — | — | — | — | — | — | — | |
| Supervised BaselineDistortion=Gaussian Noise sigma = 0.1, Labeled Data fraction=100%, Evaluation protocol=Linear probe2021.10 | 82.23 | — | — | — | — | — | — | — | — | |
| Vim-TArch.=SSM, #Param. (M)=7M2026.03 | 82.2 | 95.6 | — | — | — | — | — | — | — | |
| Robust EncoderDistortion=Random Mask 50%, Labeled Data fraction=10%, Evaluation protocol=Linear probe2021.10 | 82.19 | — | — | — | — | — | — | — | — | |
| asymmetric knowledge distillation frameworkDistortion=Random Mask 90%2026.04 | 81.94 | — | — | — | — | — | — | — | — | |
| FullIPC (Ratio)=10 (0.8%), Test Model=ResNet-182023.11 | 81.8 | — | — | — | — | — | — | — | — | |
| FullIPC (Ratio)=20 (1.6%), Test Model=ResNet-182023.11 | 81.8 | — | — | — | — | — | — | — | — | |
| SimSiamBackbone=ResNet-50, Evaluation Protocol=Linear Evaluation2022.09 | 81.6 | — | — | — | — | — | — | — | — | |
| LUQBackbone=ResNet-502025.03 | 81.45 | — | — | — | — | — | — | — | — | |
| Robust EncoderDistortion=Gaussian Noise sigma = 0.3, Labeled Data fraction=100%, Evaluation protocol=Linear probe2021.10 | 81.3 | — | — | — | — | — | — | — | — | |
| INT4Backbone=EfficientFormer-L12025.03 | 81.3 | — | — | — | — | — | — | — | — | |
| Robust EncoderDistortion=Gaussian Noise sigma = 0.1, Labeled Data fraction=10%, Evaluation protocol=Linear probe2021.10 | 80.99 | — | — | — | — | — | — | — | — | |
| Robust EncoderDistortion=Gaussian Blur n = 21, Labeled Data fraction=10%, Evaluation protocol=Linear probe2021.10 | 80.94 | — | — | — | — | — | — | — | — | |
| ReLU-Top-kBackbone=ViT-B/16, Tokens=All, Layer Range=All2026.04 | 80.94 | — | — | — | — | — | — | — | — | |
| BBQ-VisionModel=DEIT-S, Params=20M, Bits WA=32026.03 | 80.88 | — | — | 2.91 | 2.48 | — | — | — | — | |
| CorInfoMaxBackbone=ResNet-18, Evaluation Protocol=Linear Evaluation2022.09 | 80.48 | — | — | — | — | — | — | — | — | |
| Baseline (Original Model)Backbone=ViT-B/322026.04 | 80.42 | — | — | — | — | — | — | — | — | |
| BarlowBackbone=ResNet-18, Evaluation Protocol=Linear Evaluation2022.09 | 80.38 | — | — | — | — | — | — | — | — | |
| Robust EncoderDistortion=Random Mask 75%, Labeled Data fraction=10%, Evaluation protocol=Linear probe2021.10 | 80.36 | — | — | — | — | — | — | — | — | |
| BYOLBackbone=ResNet-18, Evaluation Protocol=Linear Evaluation2022.09 | 80.32 | — | — | — | — | — | — | — | — | |
| FullIPC (Ratio)=10 (0.8%), Test Model=ResNetAP-102023.11 | 80.3 | — | — | — | — | — | — | — | — | |
| FullIPC (Ratio)=20 (1.6%), Test Model=ResNetAP-102023.11 | 80.3 | — | — | — | — | — | — | — | — | |
| Vim-SArch.=SSM, #Param. (M)=26M2026.03 | 80.24 | 95 | — | — | — | — | — | — | — | |
| Supervised BaselineDistortion=Gaussian Blur n = 372026.04 | 80.08 | — | — | — | — | — | — | — | — | |
| LSQModel=DEIT-S, Params=20M, Bits WA=42026.03 | 80.04 | — | — | 3.57 | 2.52 | — | — | — | — | |
| QuESTModel=DEIT-S, Params=20M, Bits WA=42026.03 | 80 | — | — | 3.61 | 2.51 | — | — | — | — | |
| FullIPC (Ratio)=10 (0.8%), Test Model=ConvNet-62023.11 | 79.9 | — | — | — | — | — | — | — | — | |
| FullIPC (Ratio)=20 (1.6%), Test Model=ConvNet-62023.11 | 79.9 | — | — | — | — | — | — | — | — | |
| Robust EncoderDistortion=Random Mask 90%, Labeled Data fraction=10%, Evaluation protocol=Linear probe2021.10 | 79.87 | — | — | — | — | — | — | — | — | |
| HO-HEBackbone=ResNet-18, Synthetic Data Size=50k, Source Pool Size=100k2026.07 | 79.48 | — | — | — | — | — | — | — | 50 | |
| BBQ-VisionModel=DEIT-S, Params=20M, Bits WA=42026.03 | 79.46 | — | — | 3.82 | 2.53 | — | — | — | — |