Image Classification on ImageNet fine-tuning original
90.88Top-1 AccuracyCoAtNet-7
Evaluation results
| Method | Links | |
|---|---|---|
| CoAtNet-7Eval Size=512x512, Number of Parameters=2.44B, FLOPs=2586B, TPUv3-core-days=20.1K, Pre-training Dataset=JFT-3B2021.06 | 90.88 | |
| ViT-G/14Eval Size=518x518, Number of Parameters=1.84B, FLOPs=5160B, TPUv3-core-days=>30K, Pre-training Dataset=JFT-3B2021.06 | 90.45 | |
| CoAtNet-6Eval Size=512x512, Number of Parameters=1.47B, FLOPs=1521B, TPUv3-core-days=6.6K, Pre-training Dataset=JFT-3B2021.06 | 90.45 | |
| CoAtNet-5Eval Size=512x512, Number of Parameters=688M, FLOPs=812B, TPUv3-core-days=1.82K, Pre-training Dataset=JFT-300M2021.06 | 89.77 | |
| BEIT-L+Model Size=307M, Labeled Data Size=70M, Resolution=512x512, Pre-training Strategy=Self-Supervised Pre-Training and Intermediate Fine-Tuning on In-House-70M2021.06 | 89.5 | |
| BEIT-L+Model Size=307M, Labeled Data Size=70M, Resolution=384x384, Pre-training Strategy=Self-Supervised Pre-Training and Intermediate Fine-Tuning on In-House-70M2021.06 | 89.3 | |
| NFNet-F4+Eval Size=512x512, Number of Parameters=527M, FLOPs=367B, TPUv3-core-days=1.86K, Pre-training Dataset=JFT-300M2021.06 | 89.2 | |
| CoAtNet-4Eval Size=512x512, Number of Parameters=275M, FLOPs=361B, TPUv3-core-days=0.95K, Pre-training Dataset=JFT-300M2021.06 | 89.11 | |
| CoAtNet-3Eval Size=512x512, Number of Parameters=168M, FLOPs=214B, TPUv3-core-days=0.58K, Pre-training Dataset=JFT-300M, Down-sampling in MBConv=Stride-2 Depthwise Convolution2021.06 | 88.81 | |
| BEIT-L+Model Size=307M, Labeled Data Size=14M, Resolution=512x512, Pre-training Strategy=Self-Supervised Pre-Training and Intermediate Fine-Tuning on ImageNet-22K2021.06 | 88.6 | |
| ViT-H [DBK+20]Model Size=632M, Labeled Data Size=300M, Resolution=512x512, Pre-training Strategy=Supervised Pre-Training on Google JFT-300M2021.06 | 88.55 | |
| ViT-H/14Eval Size=518x518, Number of Parameters=632M, FLOPs=1021B, TPUv3-core-days=2.5K, Pre-training Dataset=JFT-300M2021.06 | 88.55 | |
| CoAtNet-3Eval Size=384x384, Number of Parameters=168M, FLOPs=114B, TPUv3-core-days=0.58K, Pre-training Dataset=JFT-300M, Down-sampling in MBConv=Stride-2 Depthwise Convolution2021.06 | 88.52 | |
| ViT-L [ZKHB21]Model Size=307M, Labeled Data Size=3000M, Resolution=384x384, Pre-training Strategy=Supervised Pre-Training on Google JFT-3B2021.06 | 88.5 | |
| BEIT-L+Model Size=307M, Labeled Data Size=14M, Resolution=384x384, Pre-training Strategy=Self-Supervised Pre-Training and Intermediate Fine-Tuning on ImageNet-22K2021.06 | 88.4 | |
| ViT-H [DBK+20]Model Size=632M, Labeled Data Size=300M, Resolution=384x384, Pre-training Strategy=Supervised Pre-Training on Google JFT-300M2021.06 | 88 | |
| ViT-L [DBK+20]Model Size=307M, Labeled Data Size=300M, Resolution=512x512, Pre-training Strategy=Supervised Pre-Training on Google JFT-300M2021.06 | 87.76 | |
| ViT-L/16Eval Size=512x512, Number of Parameters=307M, FLOPs=364B, TPUv3-core-days=0.68K, Pre-training Dataset=JFT-300M2021.06 | 87.76 | |
| ResNet + ViT-L/16Eval Size=384x384, Number of Parameters=330M, Pre-training Dataset=JFT-300M2021.06 | 87.12 | |
| ViT-L [DBK+20]Model Size=307M, Labeled Data Size=300M, Resolution=384x384, Pre-training Strategy=Supervised Pre-Training on Google JFT-300M2021.06 | 87.1 | |
| BEIT-B+Model Size=86M, Labeled Data Size=14M, Resolution=384x384, Pre-training Strategy=Self-Supervised Pre-Training and Intermediate Fine-Tuning on ImageNet-22K2021.06 | 86.8 | |
| ViT-B [ZKHB21]Model Size=86M, Labeled Data Size=3000M, Resolution=384x384, Pre-training Strategy=Supervised Pre-Training on Google JFT-3B2021.06 | 86.6 | |
| 16-TokenLearner B/16GFLOPS=47.7, Number of transformer layers=21, Number of tokens=16, Fine-tuning=true, JFT-300M pre-training=true2021.06 | 85.45 | |
| ViT-L [DBK+20]Model Size=307M, Labeled Data Size=14M, Resolution=512x512, Pre-training Strategy=Supervised Pre-Training on ImageNet-22K2021.06 | 85.3 | |
| TokenLearner B/16GFLOPS=47.1, Number of transformer layers=21, Number of tokens=8, Fine-tuning=true, JFT-300M pre-training=true2021.06 | 85.21 | |
| ViT-L [DBK+20]Model Size=307M, Labeled Data Size=14M, Resolution=384x384, Pre-training Strategy=Supervised Pre-Training on ImageNet-22K2021.06 | 85.2 | |
| ViT-H [DBK+20]Model Size=632M, Labeled Data Size=14M, Resolution=384x384, Pre-training Strategy=Supervised Pre-Training on ImageNet-22K2021.06 | 85.1 | |
| ViT B/16GFLOPS=55.6, Fine-tuning=true, JFT-300M pre-training=true2021.06 | 84.73 | |
| ViT-B [DBK+20]Model Size=86M, Labeled Data Size=300M, Resolution=384x384, Pre-training Strategy=Supervised Pre-Training on Google JFT-300M2021.06 | 84.2 | |
| ViT-B [DBK+20]Model Size=86M, Labeled Data Size=14M, Resolution=384x384, Pre-training Strategy=Supervised Pre-Training on ImageNet-22K2021.06 | 84 | |
| TokenLearner B/16GFLOPS=28.7, Number of tokens=8, Fine-tuning=true, JFT-300M pre-training=true2021.06 | 83.65 | |
| TokenLearner B/32GFLOPS=11.5, Number of transformer layers=20, Number of tokens=8, Fine-tuning=true, JFT-300M pre-training=true2021.06 | 82.74 | |
| ViT B/32GFLOPS=19.8, Fine-tuning=true, JFT-300M pre-training=true2021.06 | 80.69 | |
| TokenLearner S/32GFLOPS=3.3, Number of transformer layers=22, Number of tokens=8, Fine-tuning=true, JFT-300M pre-training=true2021.06 | 79.42 | |
| ViT S/32GFLOPS=3.4, Fine-tuning=true, JFT-300M pre-training=true2021.06 | 77.87 | |
| TokenLearner S/32GFLOPS=1.9, Number of tokens=8, Fine-tuning=true, JFT-300M pre-training=true2021.06 | 76.13 |