Image Classification on ImageNet1K 1.0 (train val)
80.4Top-1 AccMR BarTwins
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MR BarTwinsArch=ViT-B, Params.(M)=113, Pretraining-epochs=600, Supervision=RGB, α=7, λ=22022.11 | 80.4 | — | — | — | — | — | |
| DINOArch=ViT-B, Params.(M)=88, Pretraining-epochs=300, Supervision=RGB2022.11 | 80.1 | — | — | — | — | — | |
| MR SimCLRArch=ViT-B, Params.(M)=113, Pretraining-epochs=600, Supervision=RGB, α=8, λ=52022.11 | 80 | — | — | — | — | — | |
| iBOTArch=ViT-B, Params.(M)=88, Pretraining-epochs=1600, Supervision=RGB2022.11 | 79.5 | — | — | — | — | — | |
| MoCo-v3Arch=ViT-B, Params.(M)=88, Pretraining-epochs=600, Supervision=RGB2022.11 | 76.5 | — | — | — | — | — | |
| MAEArch=ViT-L, Params.(M)=218, Pretraining-epochs=1600, Supervision=RGB2022.11 | 75.1 | — | — | — | — | — | |
| BYOLArch=ResNet-50, Params.(M)=23, Pretraining-epochs=1000, Supervision=RGB2022.11 | 74.3 | — | — | — | — | — | |
| BarTwins(w/ ViT)Arch=ViT-B, Params.(M)=87, Pretraining-epochs=600, Supervision=RGB2022.11 | 74.1 | — | — | — | — | — | |
| BarTwinsArch=ResNet-50, Params.(M)=23, Pretraining-epochs=1000, Supervision=RGB2022.11 | 73.2 | — | — | — | — | — | |
| SimCLR(w/ ViT)Arch=ViT-B, Params.(M)=87, Pretraining-epochs=600, Supervision=RGB2022.11 | 73.1 | — | — | — | — | — | |
| InfoMinArch=ResNet-50, Params.(M)=23, Pretraining-epochs=800, Supervision=RGB2022.11 | 73 | — | — | — | — | — | |
| MoCo-v2Arch=ResNet-50, Params.(M)=23, Pretraining-epochs=800, Supervision=RGB2022.11 | 71.1 | — | — | — | — | — | |
| SimCLRArch=ResNet-50, Params.(M)=23, Pretraining-epochs=1000, Supervision=RGB2022.11 | 69.3 | — | — | — | — | — | |
| CAEArch=ViT-B, Params.(M)=88, Pretraining-epochs=600, Supervision=DALLE+RGB2022.11 | 68.3 | — | — | — | — | — | |
| MaskFeatArch=ViT-L, Params.(M)=218, Pretraining-epochs=1600, Supervision=HOG2022.11 | 67.7 | — | — | — | — | — | |
| InfoCLArch=ResNet-50, Params.(M)=23, Pretraining-epochs=200, Supervision=RGB2022.11 | 61.6 | — | — | — | — | — | |
| BEITArch=ViT-B, Params.(M)=86, Pretraining-epochs=800, Supervision=RGB2022.11 | 56.7 | — | — | — | — | — | |
| SimMIMArch=ViT-B, Params.(M)=88, Pretraining-epochs=800, Supervision=RGB2022.11 | 56.7 | — | — | — | — | — | |
| BaselineModel=DeiT-tiny, Hyper?=-, Architecture?=-, Strategy?=-2024.03 | — | 72.2 | — | 3,300 | 54.6 | 1 | |
| BaselineModel=DeiT-small, Hyper?=-, Architecture?=-, Strategy?=-2024.03 | — | 79.8 | — | 13,000 | 124.5 | 1 | |
| BaselineModel=DeiT-base, Hyper?=-, Architecture?=-, Strategy?=-2024.03 | — | 81.8 | — | 52,000 | 292.8 | 1 | |
| NetworkExpansion 6→12Model=DeiT-tiny, Hyper?=✓, Architecture?=✓, Strategy?=x2024.03 | — | 70.3 | 70.1 | 2,500 | 43.2 | 1.26 | |
| NetworkExpansion 6→12Model=DeiT-small, Hyper?=✓, Architecture?=✓, Strategy?=x2024.03 | — | 78.8 | 78.6 | 9,800 | 100.3 | 1.24 | |
| NetworkExpansion 6→12Model=DeiT-base, Hyper?=✓, Architecture?=✓, Strategy?=x2024.03 | — | 81 | 81.5 | 39,000 | 226.8 | 1.29 | |
| S2ViTE-TinyModel=DeiT-tiny, Hyper?=x, Architecture?=x, Strategy?=✓2024.03 | — | 70.1 | — | 2,500 | — | 1.19 | |
| StackBERTModel=DeiT-base, Hyper?=✓, Architecture?=✓, Strategy?=x2024.03 | — | 80.8 | — | 42,000 | 231.6 | 1.26 | |
| ToE Hyper r1=0.4Model=DeiT-base, Hyper?=x, Architecture?=✓, Strategy?=✓2024.03 | — | 81.7 | — | 33,000 | 202.8 | 1.44 | |
| ToE Hyper r1=0.5Model=DeiT-base, Hyper?=x, Architecture?=✓, Strategy?=✓2024.03 | — | 81.8 | — | 36,000 | 213.2 | 1.37 | |
| ToE r1=0.4Model=DeiT-base, Hyper?=✓, Architecture?=✓, Strategy?=✓2024.03 | — | 81.4 | — | 38,000 | 225.2 | 1.3 | |
| ToE r1=0.5Model=DeiT-tiny, Hyper?=✓, Architecture?=✓, Strategy?=✓2024.03 | — | 72.6 | — | 2,600 | 44.2 | 1.24 | |
| ToE r1=0.5Model=DeiT-small, Hyper?=✓, Architecture?=✓, Strategy?=✓2024.03 | — | 79.8 | — | 10,000 | 102.2 | 1.22 | |
| ToE r1=0.5Model=DeiT-base, Hyper?=✓, Architecture?=✓, Strategy?=✓2024.03 | — | 81.6 | — | 40,000 | 231.2 | 1.27 | |
| ToMeDeiT r8→Model=DeiT-tiny, Hyper?=✓, Architecture?=x, Strategy?=✓2024.03 | — | 71.7 | — | 2,500 | 53.3 | 1.02 | |
| ToMeDeiT r8→Model=DeiT-small, Hyper?=✓, Architecture?=x, Strategy?=✓2024.03 | — | 79.7 | — | 9,800 | 121.5 | 1.02 |