Image Classification on ImageNet (val) (Top-1/Top-5 Accuracy)
89.6Top-1 AccuracyBEIT-3
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| BEIT-3Model=BEIT-3 (2B), Pipeline=Stage 1: CLIP, Stage 2: Dense Distillation, Stage 3: Masked Data Modeling, Public Data=21M image-text, 15M image-category, Private Data=400M image-text2022.11 | 89.6 | — | — | — | — | — | |
| SwinV2-GModel=SwinV2-G (3B), Pipeline=Stage 1: Masked Image Modeling pixel, Stage 2: Image Classification, Stage 3: Dense Distillation, Public Data=15M image-category, Private Data=55M image-category2022.11 | 89.4 | — | — | — | — | — | |
| M3I Pre-trainingModel=InternImage-H (1B), Pipeline=Single Stage: M3I Pre-training, Public Data=427M image-text, 15M image-category2022.11 | 89.2 | — | — | — | — | — | |
| SwinV2-GModel=SwinV2-G (3B), Pipeline=Stage 1: Masked Image Modeling pixel, Stage 2: Image Classification, Public Data=15M image-category, Private Data=55M image-category2022.11 | 89.2 | — | — | — | — | — | |
| MOATData Constraints=Only public training data2022.11 | 89.1 | — | — | — | — | — | |
| HVTVersion=Huge2024.09 | 87.4 | 97.6 | — | — | — | — | |
| OriginalBackbone=AR-L, Annotated segmentation=false2022.06 | 85.6 | 97.8 | — | — | — | — | |
| RobustViTBackbone=AR-L, Annotated segmentation=false2022.06 | 85.4 | 97.6 | — | — | — | — | |
| RobustViTBackbone=AR-L, Annotated segmentation=true2022.06 | 85.1 | 97.5 | — | — | — | — | |
| HVTVersion=Large2024.09 | 85 | 96.8 | — | — | — | — | |
| Right for the Right Reason (RRR)Backbone=AR-B, Annotated segmentation=true2022.06 | 84.6 | 97.3 | — | — | — | — | |
| GradMaskBackbone=AR-B, Annotated segmentation=true2022.06 | 84.5 | 97.3 | — | — | — | — | |
| OriginalBackbone=AR-B, Annotated segmentation=false2022.06 | 84.4 | 97.2 | — | — | — | — | |
| ViTVersion=Huge2024.09 | 84.2 | 96.7 | — | — | — | — | |
| ASF-former-BType=Hybrid, Size Category=Big, Image Size=224, Params (M)=56.7, MACs (G)=12.92022.04 | 83.9 | 96.6 | — | — | — | — | |
| ASF-formerp-BType=Hybrid, Size Category=Big, Image Size=224, Params (M)=58.9, MACs (G)=12.1, Structure=Pyramid2022.04 | 83.9 | 96.5 | — | — | — | — | |
| ConvNeXt-BType=CNN, Size Category=Big, Image Size=224, Params (M)=89, MACs (G)=15.42022.04 | 83.8 | — | — | — | — | — | |
| Focal-BType=Hybrid, Size Category=Big, Image Size=224, Params (M)=89.8, MACs (G)=162022.04 | 83.8 | — | — | — | — | — | |
| RobustViTBackbone=AR-B, Annotated segmentation=false2022.06 | 83.6 | 97.1 | — | — | — | — | |
| Swin-BType=Transformer, Size Category=Big, Image Size=224, Params (M)=88, MACs (G)=15.42022.04 | 83.5 | 96.5 | — | — | — | — | |
| Swin-ACmix-SType=Hybrid, Size Category=Big, Image Size=224, Params (M)=51, MACs (G)=92022.04 | 83.5 | — | — | — | — | — | |
| Conformer-SType=Hybrid, Size Category=Big, Image Size=224, Params (M)=37.7, MACs (G)=10.62022.04 | 83.4 | — | — | — | — | — | |
| RobustViTBackbone=AR-B, Annotated segmentation=true2022.06 | 83.1 | 96.9 | — | — | — | — | |
| ASF-formerp-SType=Hybrid, Size Category=Small, Image Size=224, Params (M)=21.3, MACs (G)=5.4, Structure=Pyramid2022.04 | 83 | 96.3 | — | — | — | — | |
| OriginalBackbone=ViT-L, Annotated segmentation=false2022.06 | 82.9 | 96.4 | — | — | — | — | |
| RegNetY-16GType=CNN, Size Category=Big, Image Size=224, Params (M)=84, MACs (G)=162022.04 | 82.9 | — | — | — | — | — | |
| TNT-BType=Transformer, Size Category=Big, Image Size=224, Params (M)=65.6, MACs (G)=14.12022.04 | 82.9 | 96.3 | — | — | — | — | |
| RobustViTBackbone=ViT-L, Annotated segmentation=false2022.06 | 82.7 | 96.4 | — | — | — | — | |
| ASF-former-SType=Hybrid, Size Category=Small, Image Size=224, Params (M)=19.3, MACs (G)=5.52022.04 | 82.7 | 96.1 | — | — | — | — | |
| T2T-ViTt-24Type=Transformer, Size Category=Big, Image Size=224, Params (M)=64.1, MACs (G)=152022.04 | 82.6 | — | — | — | — | — | |
| CvT-21Type=Hybrid, Size Category=Small, Image Size=224, Params (M)=32, MACs (G)=7.12022.04 | 82.5 | — | — | — | — | — | |
| ViTVersion=Large2024.09 | 82.5 | 95.8 | — | — | — | — | |
| MEGAImg. size=224^2, #Param.=90M2022.09 | 82.3 | — | — | — | — | — | |
| T2T-ViT-24Type=Transformer, Size Category=Big, Image Size=224, Params (M)=64.1, MACs (G)=14.12022.04 | 82.3 | — | — | — | — | — | |
| HATBackbone=ViP-Small/7, Training epochs=300 (200 adversarial), Strategy=HAT2022.04 | 82.2 | — | — | — | — | — | |
| Focal-TType=Hybrid, Size Category=Small, Image Size=224, Params (M)=29.1, MACs (G)=4.92022.04 | 82.2 | — | — | — | — | — | |
| ConvNeXt-TType=CNN, Size Category=Small, Image Size=224, Params (M)=29, MACs (G)=4.52022.04 | 82.1 | — | — | — | — | — | |
| RobustViTBackbone=ViT-L, Annotated segmentation=true2022.06 | 82 | 96.2 | — | — | — | — | |
| CeiT-SType=Hybrid, Size Category=Small, Image Size=224, Params (M)=24.2, MACs (G)=4.52022.04 | 82 | 95.9 | — | — | — | — | |
| ViTAE-SType=Hybrid, Size Category=Small, Image Size=224, Params (M)=23.6, MACs (G)=5.62022.04 | 82 | 95.9 | — | — | — | — | |
| DAT-TType=Hybrid, Size Category=Small, Image Size=224, Params (M)=29, MACs (G)=4.62022.04 | 82 | — | — | — | — | — | |
| Right for the Right Reason (RRR)Backbone=ViT-B, Annotated segmentation=true2022.06 | 81.9 | 96.2 | — | — | — | — | |
| Swin-ACmix-TType=Hybrid, Size Category=Small, Image Size=224, Params (M)=30, MACs (G)=4.62022.04 | 81.9 | — | — | — | — | — | |
| GradMaskBackbone=ViT-B, Annotated segmentation=true2022.06 | 81.8 | 96.1 | — | — | — | — | |
| DeiT-BImg. size=224^2, #Param.=86M2022.09 | 81.8 | — | — | — | — | — | |
| ConT-BType=Hybrid, Size Category=Small, Image Size=224, Params (M)=39.6, MACs (G)=6.42022.04 | 81.8 | — | — | — | — | — | |
| DeiT-BType=Transformer, Size Category=Big, Image Size=224, Params (M)=86.6, MACs (G)=34.62022.04 | 81.8 | 95.6 | — | — | — | — | |
| Wide ResNet-50-2Backbone=Wide ResNet-50-22022.11 | 81.76 | 95.74 | — | — | — | — | |
| T2T-ViTt-14Type=Transformer, Size Category=Small, Image Size=224, Params (M)=21.5, MACs (G)=6.12022.04 | 81.7 | — | — | — | — | — | |
| PVT-LType=Transformer, Size Category=Big, Image Size=224, Params (M)=61.4, MACs (G)=19.62022.04 | 81.7 | — | — | — | — | — | |
| ViP-Small/7Backbone=ViP-Small/7, Training epochs=300, Strategy=Normal Training2022.04 | 81.6 | — | — | — | — | — | |
| CvT-13Type=Hybrid, Size Category=Small, Image Size=224, Params (M)=20, MACs (G)=4.52022.04 | 81.6 | — | — | — | — | — | |
| OriginalBackbone=ViT-B, Annotated segmentation=false2022.06 | 81.5 | 96 | — | — | — | — | |
| Right for the Right Reason (RRR)Backbone=AR-S, Annotated segmentation=true2022.06 | 81.5 | 96.1 | — | — | — | — | |
| TNT-SType=Transformer, Size Category=Small, Image Size=224, Params (M)=23.8, MACs (G)=5.22022.04 | 81.5 | 95.7 | — | — | — | — | |
| T2T-ViT-14Type=Transformer, Size Category=Small, Image Size=224, Params (M)=21.5, MACs (G)=5.22022.04 | 81.5 | 95.7 | — | — | — | — | |
| OriginalBackbone=AR-S, Annotated segmentation=false2022.06 | 81.4 | 96.1 | — | — | — | — | |
| 3x3 convs→SMOESMOE Placement=replace 3x3 convolutions2022.11 | 81.33 | 95.52 | — | — | — | — | |
| GradMaskBackbone=AR-S, Annotated segmentation=true2022.06 | 81.3 | 96.1 | — | — | — | — | |
| Conformer-TiType=Hybrid, Size Category=Small, Image Size=224, Params (M)=23.5, MACs (G)=5.22022.04 | 81.3 | — | — | — | — | — | |
| Swin-TType=Transformer, Size Category=Small, Image Size=224, Params (M)=28, MACs (G)=4.52022.04 | 81.2 | 95.5 | — | — | — | — | |
| GreedyNASv2-LSearch Space=ResNet search space, FLOPs (M)=4098, Params (M)=26.9, Training epochs=57, Training cost (GPU days)=9, Search number=5002021.11 | 81.1 | 95.4 | — | — | — | — | |
| GradMaskBackbone=DeiT-B, Annotated segmentation=true2022.06 | 81.1 | 95.3 | — | — | — | — | |
| Right for the Right Reason (RRR)Backbone=DeiT-B, Annotated segmentation=true2022.06 | 81 | 95.2 | — | — | — | — | |
| Last layer SMOESMOE Placement=last layer2022.11 | 80.91 | 95.42 | — | — | — | — | |
| LRLCNLayer Type=LRLCN2022.11 | 80.9 | 95.41 | — | — | — | — | |
| SMOE after first layerSMOE Placement=after first layer2022.11 | 80.85 | 95.4 | — | — | — | — | |
| ResNet-50Backbone=ResNet-502022.11 | 80.83 | 95.39 | — | — | — | — | |
| OriginalBackbone=DeiT-B, Annotated segmentation=false2022.06 | 80.8 | 94.2 | — | — | — | — | |
| SE-ResNet-50Search Space=ResNet search space, Training Strategy=same strategy as GreedyNASv2-L, FLOPs (M)=4094, Params (M)=30.62021.11 | 80.5 | 94.8 | — | — | — | — | |
| RobustViTBackbone=DeiT-B, Annotated segmentation=true2022.06 | 80.5 | 94.9 | — | — | — | — | |
| RobustViTBackbone=DeiT-B, Annotated segmentation=false2022.06 | 80.5 | 95 | — | — | — | — | |
| RobustViTBackbone=ViT-B, Annotated segmentation=false2022.06 | 80.4 | 95.4 | — | — | — | — | |
| RobustViTBackbone=ViT-B, Annotated segmentation=true2022.06 | 80.3 | 95.4 | — | — | — | — | |
| RobustViTBackbone=AR-S, Annotated segmentation=false2022.06 | 80.3 | 95.8 | — | — | — | — | |
| HATBackbone=ResNet-50, Training epochs=800 (600 adversarial), Strategy=HAT2022.04 | 80.2 | — | — | — | — | — | |
| ConT-MType=Hybrid, Size Category=Small, Image Size=224, Params (M)=19.2, MACs (G)=3.12022.04 | 80.2 | — | — | — | — | — | |
| HVTVersion=Base2024.09 | 80.1 | 95.1 | — | — | — | — | |
| RegNetY-4GType=CNN, Size Category=Small, Image Size=224, Params (M)=21, MACs (G)=42022.04 | 80 | — | — | — | — | — | |
| DeiT-SType=Transformer, Size Category=Small, Image Size=224, Params (M)=22, MACs (G)=4.62022.04 | 79.9 | 95 | — | — | — | — | |
| RobustViTBackbone=AR-S, Annotated segmentation=true2022.06 | 79.8 | 95.7 | — | — | — | — | |
| ResNet-50Backbone=ResNet-50, Training epochs=800, Strategy=Normal Training2022.04 | 79.8 | — | — | — | — | — | |
| PVT-SType=Transformer, Size Category=Small, Image Size=224, Params (M)=24.5, MACs (G)=7.62022.04 | 79.8 | — | — | — | — | — | |
| MobileOneVariant=S4, #Params=14.82022.06 | 79.4 | — | — | — | — | — | |
| SKNet-50Search Space=ResNet search space, FLOPs (M)=4470, Params (M)=27.52021.11 | 79.2 | — | — | — | — | — | |
| SE-ResNeXt-50Search Space=ResNet search space, FLOPs (M)=4233, Params (M)=27.62021.11 | 78.9 | 94.5 | — | — | — | — | |
| ResNet-152Type=CNN, Size Category=Big, Image Size=224, Params (M)=60.2, MACs (G)=22.62022.04 | 78.9 | 94.4 | — | — | — | — | |
| ResNet-50Search Space=ResNet search space, Training Strategy=same strategy as GreedyNASv2-L, FLOPs (M)=4089, Params (M)=25.62021.11 | 78.8 | 94.6 | — | — | — | — | |
| RegNetX-4.0GFSearch Space=ResNet search space, FLOPs (M)=3964, Params (M)=22.12021.11 | 78.6 | — | — | — | — | — | |
| RobustViTBackbone=DeiT-S, Annotated segmentation=true2022.06 | 78.6 | 94.5 | — | — | — | — | |
| RobustViTBackbone=DeiT-S, Annotated segmentation=false2022.06 | 78.6 | 94.4 | — | — | — | — | |
| Stimulative TrainingBackbone=ResNet-101, Evaluation Target=Main Network2022.10 | 78.58 | — | — | — | — | — | |
| ResNet-152Img. size=224^2, #Param.=60M2022.09 | 78.3 | — | — | — | — | — | |
| ResNet-101Type=CNN, Size Category=Big, Image Size=224, Params (M)=44.5, MACs (G)=15.22022.04 | 78.3 | 94.1 | — | — | — | — | |
| OriginalBackbone=DeiT-S, Annotated segmentation=false2022.06 | 78.1 | 93.7 | — | — | — | — | |
| Right for the Right Reason (RRR)Backbone=DeiT-S, Annotated segmentation=true2022.06 | 78.1 | 94.1 | — | — | — | — | |
| MobileOneVariant=S3, #Params=10.12022.06 | 78.1 | — | — | — | — | — | |
| ViT-BImg. size=384^2, #Param.=86M2022.09 | 77.9 | — | — | — | — | — | |
| ViT-B/16Type=Transformer, Size Category=Big, Image Size=384, Params (M)=86.5, MACs (G)=55.42022.04 | 77.9 | — | — | — | — | — | |
| ViTVersion=Base2024.09 | 77.9 | 93.8 | — | — | — | — |