Image Classification on CIFAR-10 (val)
99.3Top-1 AccuracyDeiT III-B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| DeiT III-BBackbone=ViT-B, Pre-training=ImageNet-1k, Epochs=400, Resolution=224x224, Crop ratio=1.02022.04 | 99.3 | — | — | — | — | — | |
| DeiT III-LBackbone=ViT-L, Pre-training=ImageNet-1k, Epochs=400, Resolution=224x224, Crop ratio=1.02022.04 | 99.3 | — | — | — | — | — | |
| AdamModel=ViT-large, Learning Rate=lr << 12026.03 | 99.28 | — | — | — | — | — | |
| DeiT-BPre-training=ImageNet-1k, Crop ratio=0.8752022.04 | 99.1 | — | — | — | — | — | |
| SGDModel=ViT-large, Learning Rate=lr > 02026.03 | 99.07 | — | — | — | — | — | |
| DeiT III-SBackbone=ViT-S, Pre-training=ImageNet-1k, Epochs=400, Resolution=224x224, Crop ratio=1.02022.04 | 98.9 | — | — | — | — | — | |
| ViT-B/16 [13]Pre-training=ImageNet-1k, Crop ratio=0.8752022.04 | 98.1 | — | — | — | — | — | |
| CCT-7/3x1*# Params=3.76 M, MACs=1.19 G, Training Duration=5000 epochs, Pos. Emb.=Sinusoidal2021.04 | 98 | — | — | — | — | — | |
| Proxyless-G# Params=5.7 M2021.04 | 97.92 | — | — | — | — | — | |
| ViT-L/16 [13]Pre-training=ImageNet-1k, Crop ratio=0.8752022.04 | 97.9 | — | — | — | — | — | |
| Vertical TokenMixup# Params=3.78 M, MACs=0.95 G, epochs=1500, backbone=CCT-7/3x12022.10 | 97.78 | — | — | — | — | — | |
| TokenMixup# Params=3.81 M, MACs=0.95 G, epochs=1500, backbone=CCT-7/3x1, variant=HTM + VTM2022.10 | 97.75 | — | — | — | — | — | |
| Horizontal TokenMixup# Params=3.81 M, MACs=0.95 G, epochs=1500, backbone=CCT-7/3x12022.10 | 97.57 | — | — | — | — | — | |
| CCT-7/3x1# Params=3.76 M, MACs=0.95 G, epochs=15002022.10 | 97.48 | — | — | — | — | — | |
| CCT-7/3x1# Params=3.78 M, MACs=0.95 G, epochs=1500, retrained=true2022.10 | 97.48 | — | — | — | — | — | |
| NesT-B# Params=68.0 M2022.10 | 97.2 | — | — | — | — | — | |
| CCT-7/3x1# Params=3.76 M, MACs=1.19 G, Training Duration=300 epochs2021.04 | 96.53 | — | — | — | — | — | |
| ResNeXt-29-16x64d# Params=68.1 M2022.10 | 96.42 | — | — | — | — | — | |
| Mae-ViT-C10# Params=3.64 M, MACs=0.26 G2024.02 | 96.41 | — | — | — | — | — | |
| ResNeXt-29-8x64d# Params=34.4 M2022.10 | 96.35 | — | — | — | — | — | |
| Reference Masking RegularizationBackbone=ViTb16, Pre-trained=ImageNet-1K, Adaptation Protocol=Linear Probing2024.06 | 96.29 | — | — | — | — | — | |
| NesT-T# Params=17.0 M2022.10 | 96.04 | — | — | — | — | — | |
| Mae-ViT-C100# Params=3.64 M, MACs=0.26 G2024.02 | 96.01 | — | — | — | — | — | |
| WRN-28-10# Params=36.5 M2022.10 | 96 | — | — | — | — | — | |
| Zhang et al., 2018Noise level (%)=202019.04 | 95.6 | — | — | — | — | — | |
| mixupNoise level (%)=20, Evaluation Point=Best2019.04 | 95.6 | — | — | — | — | — | |
| StandardBackbone=ViTb16, Pre-trained=ImageNet-1K, Adaptation Protocol=Linear Probing2024.06 | 95.48 | — | — | — | — | — | |
| WRN-40-4# Params=8.9 M2022.10 | 95.47 | — | — | — | — | — | |
| ResNet1k-v2*# Params=10.33 M, MACs=1.55 G2021.04 | 95.38 | — | — | — | — | — | |
| ResNet1k-v2*# Params=10.33 M, MACs=1.55 G2024.02 | 95.38 | — | — | — | — | — | |
| Zhang et al., 2018Noise level (%)=02019.04 | 95.3 | — | — | — | — | — | |
| mixupNoise level (%)=0, Evaluation Point=Best2019.04 | 95.3 | — | — | — | — | — | |
| LOOKAHEADBackbone=ResNet-18, Optimizer=LOOKAHEAD2019.07 | 95.27 | — | — | — | — | — | |
| POLYAKBackbone=ResNet-18, Optimizer=POLYAK2019.07 | 95.26 | — | — | — | — | — | |
| SGDBackbone=ResNet-18, Optimizer=SGD2019.07 | 95.23 | — | — | — | — | — | |
| mixupNoise level (%)=0, Evaluation Point=Last2019.04 | 95.2 | — | — | — | — | — | |
| AdamModel=ResNet-18, Learning Rate=lr << 12026.03 | 95.19 | — | — | — | — | — | |
| ResNet110# Params=1.73 M, MACs=0.26 G2021.04 | 95.08 | — | — | — | — | — | |
| ResNet1001-v2# Params=10.33 M, MACs=1.55 G2022.10 | 95.08 | — | — | — | — | — | |
| ResNet110# Params=1.73 M, MACs=0.26 G2024.02 | 95.08 | — | — | — | — | — | |
| CCT-7/3x2# Params=3.85 M, MACs=0.29 G2021.04 | 95.04 | — | — | — | — | — | |
| CCT-7/3 x 2# Params=3.85 M, MACs=0.29 G2024.02 | 95.04 | — | — | — | — | — | |
| ADAMBackbone=ResNet-18, Optimizer=ADAM2019.07 | 94.84 | — | — | — | — | — | |
| Reed et al., 2015Noise level (%)=02019.04 | 94.7 | — | — | — | — | — | |
| Patrini et al., 2017Noise level (%)=02019.04 | 94.7 | — | — | — | — | — | |
| CENoise level (%)=0, Evaluation Point=Best2019.04 | 94.7 | — | — | — | — | — | |
| NEOLITHICBackbone=ResNet18, Number of workers=8, Compression ratio=5%2022.06 | 94.63 | — | — | — | — | — | |
| ResNet56# Params=0.85 M, MACs=0.13 G2021.04 | 94.63 | — | — | — | — | — | |
| ResNet56# Params=0.85 M, MACs=0.13 G2024.02 | 94.63 | — | — | — | — | — | |
| CENoise level (%)=0, Evaluation Point=Last2019.04 | 94.6 | — | — | — | — | — | |
| Direct ConvPrecision=FP32, Time=27.5ms2025.12 | 94.5 | — | — | — | 99.8 | — | |
| Wino StdPrecision=FP32, kappa=42.5, Time=392ms2025.12 | 94.5 | — | — | — | 99.8 | — | |
| Wino DiscPrecision=FP32, kappa=14.5, Time=391ms2025.12 | 94.5 | — | — | — | 99.8 | — | |
| MEM-SGDBackbone=ResNet18, Number of workers=8, Compression ratio=5%2022.06 | 94.35 | — | — | — | — | — | |
| NEOLITHICBackbone=ResNet-18, Number of Workers=8, Compression Ratio=1%2022.06 | 94.155 | — | — | — | — | — | |
| DOUBLE-SQUEEZEBackbone=ResNet18, Number of workers=8, Compression ratio=5%2022.06 | 94.11 | — | — | — | — | — | |
| OrScaleLR=0.022026.05 | 94.05 | — | — | — | — | — | |
| full-precisionBit-width=full-precision, Backbone=ResNet-202021.11 | 94.03 | — | — | — | — | — | |
| CVT-7/4# Params=3.72 M, MACs=0.25 G2021.04 | 94.01 | — | — | — | — | — | |
| CVT-7/4# Params=3.72 M, MACs=0.25 G2024.02 | 94.01 | — | — | — | — | — | |
| M-DYR-HNoise level (%)=202019.04 | 94 | — | — | — | — | — | |
| M-DYR-HNoise level (%)=20, Evaluation Point=Best2019.04 | 94 | — | — | — | — | — | |
| PSGDBackbone=ResNet18, Number of workers=8, Compression ratio=5%2022.06 | 93.99 | — | — | — | — | — | |
| MEM-SGDBackbone=ResNet-18, Number of Workers=8, Compression Ratio=1%2022.06 | 93.99 | — | — | — | — | — | |
| ViT-Lite-6/4# Params=3.19 M, MACs=0.22 G2022.10 | 93.98 | — | — | — | — | — | |
| Full PrecisionBackbone=ResNet-20, Quantization Bits (W/A)=W32A32, Quantization Protocol (Zero-shot)=false2026.03 | 93.89 | — | — | — | — | — | |
| MD-DYR-SHNoise level (%)=202019.04 | 93.8 | — | — | — | — | — | |
| M-DYR-HNoise level (%)=20, Evaluation Point=Last2019.04 | 93.8 | — | — | — | — | — | |
| Muon + MoonlightLR=0.012026.05 | 93.75 | — | — | — | — | — | |
| MuonLR=0.042026.05 | 93.7 | — | — | — | — | — | |
| RPGBackbone=VGG19, Sparsity=99%2023.11 | 93.62 | — | — | — | — | — | |
| M-DYR-HNoise level (%)=02019.04 | 93.6 | — | — | — | — | — | |
| MD-DYR-SHNoise level (%)=02019.04 | 93.6 | — | — | — | — | — | |
| M-DYR-HNoise level (%)=0, Evaluation Point=Best2019.04 | 93.6 | — | — | — | — | — | |
| CVT-6/4# Params=3.19 M, MACs=0.22 G2022.10 | 93.6 | — | — | — | — | — | |
| SGDModel=ResNet-18, Learning Rate=lr > 02026.03 | 93.6 | — | — | — | — | — | |
| ViT-Lite-7/4# Params=3.72 M, MACs=0.26 G2021.04 | 93.57 | — | — | — | — | — | |
| ViT-Lite-7/4# Params=3.72 M, MACs=0.26 G2022.10 | 93.57 | — | — | — | — | — | |
| ViT-Lite-7/4# Params=3.72 M, MACs=0.26 G2024.02 | 93.57 | — | — | — | — | — | |
| DOUBLE-SQUEEZEBackbone=ResNet-18, Number of Workers=8, Compression Ratio=1%2022.06 | 93.54 | — | — | — | — | — | |
| M-DYR-SNoise level (%)=20, Evaluation Point=Best2019.04 | 93.5 | — | — | — | — | — | |
| AdderQuantBackbone=VGG-Small, Bits=6, Protocol=PTQ2022.12 | 93.48 | — | — | — | — | — | |
| AdderQuantBackbone=VGG-Small, Bits=4, Protocol=QAT2022.12 | 93.46 | — | — | — | — | — | |
| AdderQuantBackbone=VGG-Small, Bits=8, Protocol=PTQ2022.12 | 93.42 | — | — | — | — | — | |
| AdderQuantBackbone=VGG-Small, Bits=5, Protocol=PTQ2022.12 | 93.41 | — | — | — | — | — | |
| M-DYR-HNoise level (%)=0, Evaluation Point=Last2019.04 | 93.4 | — | — | — | — | — | |
| DEnsBackbone=ResNet-202022.07 | 93.4 | — | — | — | — | — | |
| ProbMaskBackbone=VGG19, Sparsity=99%2023.11 | 93.38 | — | — | — | — | — | |
| AC/DCBackbone=VGG19, Sparsity=99%2023.11 | 93.35 | — | — | — | — | — | |
| M-DYR-SNoise level (%)=0, Evaluation Point=Best2019.04 | 93.3 | — | — | — | — | — | |
| AdderQuantBackbone=VGG-Small, Bits=4, Protocol=PTQ2022.12 | 93.2 | — | — | — | — | — | |
| CorInfoMaxBackbone=ResNet-18, Evaluation Protocol=Linear Evaluation2022.09 | 93.18 | — | — | — | — | — | |
| RPGBackbone=VGG19, Sparsity=99.5%2023.11 | 93.13 | — | — | — | — | — | |
| AdamWLR=0.012026.05 | 93.12 | — | — | — | — | — | |
| M-DYR-SNoise level (%)=20, Evaluation Point=Last2019.04 | 93.1 | — | — | — | — | — | |
| TNTBackbone=ResNet32, initial learning rate=1e-4, weight decay factor=10, batch size=1282021.06 | 93.08 | — | — | — | — | — | |
| SGD-mBackbone=ResNet32, initial learning rate=0.03, weight decay factor=0.01, batch size=1282021.06 | 93.06 | — | — | — | — | — | |
| M-DYR-SNoise level (%)=0, Evaluation Point=Last2019.04 | 93 | — | — | — | — | — | |
| MoCo-V2Backbone=ResNet-18, Evaluation Protocol=Linear Evaluation2022.09 | 92.94 | — | — | — | — | — | |
| AdamBackbone=ResNet32, initial learning rate=0.003, weight decay factor=0.1, batch size=1282021.06 | 92.92 | — | — | — | — | — |