Image Classification on Tiny-ImageNet (Top-1/Top-5 Accuracy)
90Top-1 AccuracyAdapterTune
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| AdapterTuneBackbone=ViT-B/162026.03 | 90 | — | — | |
| AdapterTuneBackbone=ViT-S/162026.03 | 84.95 | — | — | |
| Head-OnlyBackbone=ViT-B/162026.03 | 81.45 | — | — | |
| AdapterTuneBackbone=DeiT-T2026.03 | 76.35 | — | — | |
| Head-OnlyBackbone=ViT-S/162026.03 | 73.69 | — | — | |
| Haar Wavelet (HWT, 3-path)Evaluation Protocol=10-Crop, Params (M)=20.78 (-12.4%)2026.03 | 73.24 | 91.48 | 1.02 | |
| SAMixBackbone=ResNeXt-502021.11 | 72.18 | — | — | |
| Full FTBackbone=ViT-B/162026.03 | 71.99 | — | — | |
| Full FTBackbone=ViT-S/162026.03 | 71.38 | — | — | |
| Haar Wavelet (HWT, 3-path)Evaluation Protocol=Single-Crop, Params (M)=20.78 (-12.4%)2026.03 | 70.84 | 90.15 | 1.175 | |
| AutoMixBackbone=ResNeXt-502021.11 | 70.72 | — | — | |
| Head-OnlyBackbone=DeiT-T2026.03 | 70.28 | — | — | |
| Hadamard Transform (HT, 3-path)Evaluation Protocol=10-Crop, Params (M)=20.78 (-12.4%)2026.03 | 69.06 | 89.29 | 1.228 | |
| Full FTBackbone=DeiT-T2026.03 | 69.04 | — | — | |
| SAMixBackbone=ResNet-182021.11 | 68.89 | — | — | |
| AutoMixBackbone=ResNet-182021.11 | 67.33 | — | — | |
| ManifoldMixBackbone=ResNeXt-502021.11 | 67.3 | — | — | |
| PuzzleMixBackbone=ResNeXt-502021.11 | 66.92 | — | — | |
| Hadamard Transform (HT, 3-path)Evaluation Protocol=Single-Crop, Params (M)=20.78 (-12.4%)2026.03 | 66.65 | 87.43 | 1.39 | |
| SaliencyMixBackbone=ResNeXt-502021.11 | 66.55 | — | — | |
| CutMixBackbone=ResNeXt-502021.11 | 66.47 | — | — | |
| MixupBackbone=ResNeXt-502021.11 | 66.36 | — | — | |
| ResizeMixBackbone=ResNeXt-502021.11 | 65.87 | — | — | |
| PuzzleMixBackbone=ResNet-182021.11 | 65.81 | — | — | |
| ResNet (our trial, baseline)Evaluation Protocol=10-Crop, Params (M)=23.722026.03 | 65.68 | 87.83 | 1.288 | |
| CutMixBackbone=ResNet-182021.11 | 65.53 | — | — | |
| FMixBackbone=ResNeXt-502021.11 | 65.08 | — | — | |
| VanillaBackbone=ResNeXt-502021.11 | 65.04 | — | — | |
| SaliencyMixBackbone=ResNet-182021.11 | 64.6 | — | — | |
| ManifoldMixBackbone=ResNet-182021.11 | 64.15 | — | — | |
| MixupBackbone=ResNet-182021.11 | 63.86 | — | — | |
| ResizeMixBackbone=ResNet-182021.11 | 63.74 | — | — | |
| FMixBackbone=ResNet-182021.11 | 63.47 | — | — | |
| ResNet (our trial, baseline)Evaluation Protocol=Single-Crop, Params (M)=23.722026.03 | 63.28 | 86.37 | 1.409 | |
| VanillaBackbone=ResNet-182021.11 | 61.68 | — | — | |
| TeacherTeacher Architecture=ResNet110, Student Architecture=ResNet32, Data Size=50K~100K2022.05 | 60.71 | — | — | |
| TeacherTeacher Architecture=ResNet110, Student Architecture=VGG11, Data Size=50K~100K2022.05 | 60.71 | — | — | |
| TeacherTeacher Architecture=ResNet110, Student Architecture=MobileNet, Data Size=50K~100K2022.05 | 60.71 | — | — | |
| EL2NBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.12026.03 | 60.33 | — | — | |
| SCOPEBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.32026.03 | 60.31 | — | — | |
| MLTeacher Architecture=ResNet110, Student Architecture=MobileNet, Data Size=50K~100K2022.05 | 60.07 | — | — | |
| TeacherTeacher Architecture=VGG13, Student Architecture=VGG11, Data Size=50K~100K2022.05 | 59.89 | — | — | |
| Full DBBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.02026.03 | 59.85 | — | — | |
| FedCSBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.12026.03 | 59.79 | — | — | |
| GradNDBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.12026.03 | 59.74 | — | — | |
| GradNDBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.32026.03 | 59.49 | — | — | |
| DKDTeacher Architecture=ResNet110, Student Architecture=MobileNet, Data Size=50K~100K2022.05 | 59.43 | — | — | |
| FedCoreBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.12026.03 | 59.34 | — | — | |
| SCOPEBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.52026.03 | 59.27 | — | — | |
| SCOPEBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.12026.03 | 59.23 | — | — | |
| EL2NBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.32026.03 | 59.22 | — | — | |
| EL2NBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.52026.03 | 59.13 | — | — | |
| GradNDBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.52026.03 | 58.88 | — | — | |
| ForgettingBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.32026.03 | 58.87 | — | — | |
| SCOPEBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.72026.03 | 58.87 | — | — | |
| FedCSBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.52026.03 | 58.84 | — | — | |
| FedCSBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.32026.03 | 58.81 | — | — | |
| ForgettingBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.72026.03 | 58.8 | — | — | |
| SCOPEBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.92026.03 | 58.78 | — | — | |
| RandomBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.32026.03 | 58.68 | — | — | |
| FedCoreBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.92026.03 | 58.64 | — | — | |
| GradNDBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.72026.03 | 58.61 | — | — | |
| FedCoreBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.32026.03 | 58.57 | — | — | |
| RandomBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.12026.03 | 58.42 | — | — | |
| EL2NBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.72026.03 | 58.36 | — | — | |
| GradNDBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.92026.03 | 58.33 | — | — | |
| FedCSBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.72026.03 | 58.32 | — | — | |
| FedCSBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.92026.03 | 58.25 | — | — | |
| EL2NBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.92026.03 | 58.18 | — | — | |
| ForgettingBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.12026.03 | 58.17 | — | — | |
| ForgettingBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.52026.03 | 58.15 | — | — | |
| FedCoreBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.52026.03 | 58.13 | — | — | |
| KDTeacher Architecture=ResNet110, Student Architecture=MobileNet, Data Size=50K~100K2022.05 | 57.85 | — | — | |
| FedCoreBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.72026.03 | 57.64 | — | — | |
| MLTeacher Architecture=VGG13, Student Architecture=VGG11, Data Size=50K~100K2022.05 | 57.46 | — | — | |
| RandomBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.52026.03 | 57.36 | — | — | |
| MLTeacher Architecture=ResNet110, Student Architecture=VGG11, Data Size=50K~100K2022.05 | 56.78 | — | — | |
| RandomBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.72026.03 | 56.7 | — | — | |
| ForgettingBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.92026.03 | 56.57 | — | — | |
| MLTeacher Architecture=ResNet110, Student Architecture=ResNet32, Data Size=50K~100K2022.05 | 56.56 | — | — | |
| DKDTeacher Architecture=ResNet110, Student Architecture=VGG11, Data Size=50K~100K2022.05 | 56.52 | — | — | |
| StudentTeacher Architecture=ResNet110, Student Architecture=MobileNet, Data Size=50K~100K2022.05 | 56.07 | — | — | |
| DKDTeacher Architecture=ResNet110, Student Architecture=ResNet32, Data Size=50K~100K2022.05 | 55.99 | — | — | |
| DKDTeacher Architecture=VGG13, Student Architecture=VGG11, Data Size=50K~100K2022.05 | 55.88 | — | — | |
| RandomBackbone=ResNet-50, Imbalance Ratio (IR)=2, Label partitioning (pl)=0.1, Non-IID data distribution (α)=1.0, Pruning rate (pf)=0.92026.03 | 55.67 | — | — | |
| StudentTeacher Architecture=ResNet110, Student Architecture=ResNet32, Data Size=50K~100K2022.05 | 55.47 | — | — | |
| SCOPEBackbone=ResNet-50, Imbalance Ratio (IR)=5, Label partitioning (pl)=0.1, Non-IID data distribution (α)=0.1, Pruning rate (pf)=0.92026.03 | 55.38 | — | — | |
| MEKD (soft)Teacher Architecture=ResNet110, Student Architecture=MobileNet, Data Size=10K2022.05 | 54.93 | — | — | |
| MEKD (hard)Teacher Architecture=ResNet110, Student Architecture=MobileNet, Data Size=10K2022.05 | 54.71 | — | — | |
| AFLSetting=alpha = 0.12024.05 | 54.67 | — | — | |
| AFLSetting=alpha = 0.012024.05 | 54.67 | — | — | |
| AFLSetting=s = 102024.05 | 54.67 | — | — | |
| AFLSetting=s = 52024.05 | 54.67 | — | — | |
| SCOPEBackbone=ResNet-50, Imbalance Ratio (IR)=5, Label partitioning (pl)=0.1, Non-IID data distribution (α)=0.1, Pruning rate (pf)=0.12026.03 | 54.65 | — | — | |
| ForgettingBackbone=ResNet-50, Imbalance Ratio (IR)=5, Label partitioning (pl)=0.1, Non-IID data distribution (α)=0.1, Pruning rate (pf)=0.12026.03 | 54.63 | — | — | |
| GradNDBackbone=ResNet-50, Imbalance Ratio (IR)=5, Label partitioning (pl)=0.1, Non-IID data distribution (α)=0.1, Pruning rate (pf)=0.12026.03 | 54.63 | — | — | |
| FedCSBackbone=ResNet-50, Imbalance Ratio (IR)=5, Label partitioning (pl)=0.1, Non-IID data distribution (α)=0.1, Pruning rate (pf)=0.12026.03 | 54.6 | — | — | |
| RandomBackbone=ResNet-50, Imbalance Ratio (IR)=5, Label partitioning (pl)=0.1, Non-IID data distribution (α)=0.1, Pruning rate (pf)=0.12026.03 | 54.52 | — | — | |
| SCOPEBackbone=ResNet-50, Imbalance Ratio (IR)=5, Label partitioning (pl)=0.1, Non-IID data distribution (α)=0.1, Pruning rate (pf)=0.72026.03 | 54.49 | — | — | |
| EL2NBackbone=ResNet-50, Imbalance Ratio (IR)=5, Label partitioning (pl)=0.1, Non-IID data distribution (α)=0.1, Pruning rate (pf)=0.32026.03 | 54.44 | — | — |