Video Classification on Kinetics-400 (test)
88.9Top-1 AccCoCa ViT-giant
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CoCa ViT-giantVenue=arXiv'22, Input=6×288², Pre-training=JFT-3B+ALIGN-1.8B, Param=21002022.12 | 88.9 | — | — | |
| BIKE VIT-L/14Venue=CVPR'23, Input=32×336², Pre-training=WIT-400M, Views=4×3, FLOPs=3728×12, Param=2302022.12 | 88.6 | 98.3 | — | |
| BIKE VIT-L/14Venue=CVPR'23, Input=8×336², Pre-training=WIT-400M, Views=4×3, FLOPs=932×12, Param=2302022.12 | 88.3 | 98.1 | — | |
| BIKE VIT-L/14Venue=CVPR'23, Input=16×224², Pre-training=WIT-400M, Views=4×3, FLOPs=830×12, Param=2302022.12 | 88.1 | 97.9 | — | |
| Text4Vis ViT-L/14Venue=AAAI'23, Input=32×336², Pre-training=WIT-400M, Views=1×3, FLOPs=3829×3, Param=230.72022.12 | 87.8 | 97.6 | — | |
| Ours ViT-L/14Input=32×336², Pre-train=WIT-400M, FLOPs x Views=3829×1×3, Param=230.72022.07 | 87.8 | 97.6 | — | |
| EVL VIT-L/14Venue=ECCV'22, Input=32×336², Pre-training=WIT-400M, Views=3×1, FLOPs=181962022.12 | 87.7 | — | — | |
| X-CLIP ViT-L/14Venue=ECCV'22, Input=16×336², Pre-training=WIT-400M, Views=4×3, FLOPs=3086×122022.12 | 87.7 | 97.4 | — | |
| AIM VIT-L/14Venue=ICLR'23, Input=32×224², Pre-training=WIT-400M, Views=3×1, FLOPs=11208, Param=3412022.12 | 87.5 | 97.7 | — | |
| EVL VIT-L/14Venue=ECCV'22, Input=32×224², Pre-training=WIT-400M, Views=3×1, FLOPs=80882022.12 | 87.3 | — | — | |
| CoVeRVenue=arXiv'21, Input=16×448², Pre-training=JFT-3B, Views=1×32022.12 | 87.2 | — | — | |
| ST-Adapter ViT-L/14Venue=NeurIPS'22, Input=32×224², Pre-training=WIT-400M, Views=3×1, FLOPs=82482022.12 | 87.2 | 97.6 | — | |
| CoVeRInput=16×448², Pre-train=JF1-3B, FLOPs x Views=-×1×32022.07 | 87.2 | — | — | |
| Ours ViT-L/14Input=32×224², Pre-train=WIT-400M, FLOPs x Views=1662×4×3, Param=230.72022.07 | 87.1 | 97.4 | — | |
| FlorenceVenue=arXiv'21, Input=32×384², Pre-training=FLD-900M, Views=4×3, Param=6472022.12 | 86.5 | 97.3 | — | |
| FlorenceInput=32×384², Pre-train=FLD-900M, FLOPs x Views=-×4×3, Param=6472022.07 | 86.5 | 97.3 | — | |
| CoVeRVenue=arXiv'21, Input=16×448², Pre-training=JFT-300M, Views=1×32022.12 | 86.3 | — | — | |
| CoVeRInput=16×448², Pre-train=JFT-300M, FLOPs x Views=-×1×32022.07 | 86.3 | — | — | |
| MTV-HVenue=CVPR'22, Input=32×224², Pre-training=JFT-300M, Views=4×3, FLOPs=3706×122022.12 | 85.8 | 96.6 | — | |
| MTV-HInput=32×224², Pre-train=JFT-300M, FLOPs x Views=3706×4×32022.07 | 85.8 | 96.6 | — | |
| TokenLearner-L/10Venue=NeurIPS'21, Input=32×224², Pre-training=JFT-300M, Views=4×3, FLOPs=4076×12, Param=4502022.12 | 85.4 | 96.3 | — | |
| TokLearner-L/10Input=32×224², Pre-train=JFT-300M, FLOPs x Views=4076×4×3, Param=4502022.07 | 85.4 | 96.3 | — | |
| VideoSwin-LVenue=CVPR'22, Input=32×384², Pre-training=ImageNet-21K, Views=10×5, FLOPs=2107×50, Param=200.02022.12 | 84.9 | 96.7 | — | |
| VideoSwin-LInput=32×384², Pre-train=IN-21K, FLOPs x Views=2107×10×5, Param=200.02022.07 | 84.9 | 96.7 | — | |
| ViViT-H/16×2Venue=ICCV'21, Input=32×224², Pre-training=JFT-300M, Views=4×3, FLOPs=8316×12, Param=647.52022.12 | 84.8 | 95.8 | — | |
| ActionCLIP ViT-B/16Venue=arXiv'21, Input=32×224², Pre-training=WIT-400M, Views=10×3, FLOPs=563×30, Param=141.72022.12 | 83.8 | 96.2 | — | |
| ActionCLIPInput=32×224², Pre-train=WIT-400M, FLOPs x Views=563×10×3, Param=141.72022.07 | 83.8 | 96.2 | — | |
| ViViT-L/16×2Venue=ICCV'21, Input=32×320², Pre-training=JFT-300M, Views=4×3, FLOPs=3992×12, Param=310.82022.12 | 83.5 | 95.5 | — | |
| ViViT-L/16×2Input=32×320², Pre-train=JFT-300M, FLOPs x Views=3992×4×3, Param=310.82022.07 | 83.5 | 95.5 | — | |
| StructViT-B-4-1pretrain=IN-1K, frame x crop x clip=32x3x4, FLOPs (G)=79742024.04 | 83.4 | 95.8 | — | |
| StructViT-B-4-1pretrain=IN-1K, frame x crop x clip=32x1x4, FLOPs (G)=26582024.04 | 83.3 | 95.6 | — | |
| StructViT-B-4-2pretrain=IN-1K, frame x crop x clip=32x1x4, FLOPs (G)=10452024.04 | 83.1 | 95.5 | — | |
| UniFormer-Bpretrain=IN-1K, frame x crop x clip=32x1x4, FLOPs (G)=10362024.04 | 82.9 | 95.4 | — | |
| MViTv2-B,32x3frame x crop x clip=32x1x5, FLOPs (G)=11252024.04 | 82.9 | 95.7 | — | |
| Swin-Bpretrain=IN-21K, frame x crop x clip=32x3x4, FLOPs (G)=33842024.04 | 82.7 | 95.5 | — | |
| ip-CSN-152Input=32×224², Pre-train=IG-65M, FLOPs x Views=109×10×3, Param=32.82022.07 | 82.5 | 95.3 | — | |
| MTV-Bpretrain=IN-21K, frame x crop x clip=32x3x4, FLOPs (G)=47902024.04 | 81.8 | 95 | — | |
| StructViT-S-8-1pretrain=IN-1K, frame x crop x clip=16x1x4, FLOPs (G)=5412024.04 | 81.6 | 95.8 | — | |
| MoViNet-A6frame x crop x clip=120x1x1, FLOPs (G)=3862024.04 | 81.5 | 95.3 | — | |
| RViT-XLcausality=causal, Pre-Train=IN-21K, Param (M)=107.72024.12 | 81.5 | — | — | |
| StructViT-S-4-1pretrain=IN-1K, frame x crop x clip=16x1x4, FLOPs (G)=3272024.04 | 81.4 | 95.7 | — | |
| ViViT-L/16×2Venue=ICCV'21, Input=32×320², Pre-training=ImageNet-21K, Views=4×3, FLOPs=3992×12, Param=310.82022.12 | 81.3 | 94.7 | — | |
| ViViT-L/16×2Input=32×320², Pre-train=IN-21K, FLOPs x Views=3992×4×3, Param=310.82022.07 | 81.3 | 94.7 | — | |
| Mformer-HRpretrain=IN-21K, frame x crop x clip=16x3x10, FLOPs (G)=287642024.04 | 81.1 | 95.2 | — | |
| Dualformer-Bpretrain=IN-1K, frame x crop x clip=32x1x4, FLOPs (G)=10722024.04 | 81.1 | 95 | — | |
| StructViT-S-4-2pretrain=IN-1K, frame x crop x clip=16x1x4, FLOPs (G)=1692024.04 | 81.1 | 95.5 | — | |
| MoViNet-A5frame x crop x clip=120x1x1, FLOPs (G)=2812024.04 | 80.9 | 94.9 | — | |
| UniFormer-Spretrain=IN-1K, frame x crop x clip=16x1x4, FLOPs (G)=1672024.04 | 80.8 | 94.7 | — | |
| TimeSformer-LVenue=ICML'21, Input=96×224², Pre-training=ImageNet-21K, Views=1×3, FLOPs=2380×3, Param=121.42022.12 | 80.7 | 94.7 | — | |
| TimeSformer-LInput=96×224², Pre-train=IN-21K, FLOPs x Views=2380×1×3, Param=121.42022.07 | 80.7 | 94.7 | — | |
| TimeSformer-Lpretrain=IN-21K, frame x crop x clip=96x3x1, FLOPs (G)=71402024.04 | 80.7 | 94.7 | — | |
| ViViT-Lpretrain=IN-21K, frame x crop x clip=16x3x4, FLOPs (G)=173522024.04 | 80.6 | 94.7 | — | |
| Swin-Bpretrain=IN-1K, frame x crop x clip=32x3x4, FLOPs (G)=33842024.04 | 80.6 | 94.6 | — | |
| Dualformer-Spretrain=IN-1K, frame x crop x clip=32x1x4, FLOPs (G)=6362024.04 | 80.6 | 94.9 | — | |
| X3D-XXLInput=16×440², Pre-train=Scratch, FLOPs x Views=144×10×3, Param=20.32022.07 | 80.4 | 94.6 | — | |
| ViViT-Lcausality=non-causal, Pre-Train=IN-21K, Param (M)=320.0, dataset_version=original larger2024.12 | 80.3 | — | — | |
| X-ViTpretrain=IN-21K, frame x crop x clip=16x3x1, FLOPs (G)=8502024.04 | 80.2 | 94.7 | — | |
| MVIT-B,32x3frame x crop x clip=32x1x5, FLOPs (G)=8502024.04 | 80.2 | 94.4 | — | |
| SlowFast NL101Input=16×224², Pre-train=Scratch, FLOPs x Views=234×10×3, Param=59.92022.07 | 79.8 | 93.9 | — | |
| SlowFast+NLframe x crop x clip=16x3x10, FLOPs (G)=70202024.04 | 79.8 | 93.9 | — | |
| TimeSformer-HRpretrain=IN-21K, frame x crop x clip=16x3x1, FLOPs (G)=51092024.04 | 79.7 | 94.4 | — | |
| Mformer-Bcausality=non-causal, Pre-Train=IN-21K, Param (M)=114.02024.12 | 79.7 | — | — | |
| ip-CSNpretrain=Sports1M, frame x crop x clip=32x3x10, FLOPs (G)=32702024.04 | 79.2 | 93.8 | — | |
| MVFNet_EnVenue=AAAI'21, Input=24×224², Pre-training=ImageNet-1K, Views=10×3, FLOPs=188×302022.12 | 79.1 | 93.8 | — | |
| MVFNet_EnInput=24×224², Pre-train=IN-1K, FLOPs x Views=188×10×32022.07 | 79.1 | 93.8 | — | |
| X3D-XLframe x crop x clip=16x3x10, FLOPs (G)=14522024.04 | 79.1 | 93.9 | — | |
| ViViT-Lcausality=non-causal, Pre-Train=IN-21K, Param (M)=310.9, dataset_version=current2024.12 | 78.7 | — | — | |
| MVIT-B,16x4frame x crop x clip=16x1x5, FLOPs (G)=3532024.04 | 78.4 | 93.5 | — | |
| ViViT-Bcausality=non-causal, Pre-Train=IN-21K, Param (M)=91.2, dataset_version=current2024.12 | 78.1 | — | — | |
| TimeSformercausality=non-causal, Pre-Train=IN-21K, Param (M)=121.42024.12 | 78 | — | — | |
| NL I3D-101Venue=CVPR'18, Input=128×224², Pre-training=ImageNet-1K, Views=10×3, FLOPs=359×30, Param=61.82022.12 | 77.7 | 93.3 | — | |
| NL I3D-101Input=128×224², Pre-train=IN-1K, FLOPs x Views=359×10×3, Param=61.82022.07 | 77.7 | 93.3 | — | |
| VideoPrompt ViT-B/16Venue=ECCV'22, Input=16×224², Pre-training=WIT-400M2022.12 | 76.9 | 93.5 | — | |
| VideoPromptInput=16×224², Pre-train=WIT-400M2022.07 | 76.9 | 93.5 | — | |
| TRecViTcausality=causal, Pre-Train=IN-21K, Param (M)=111.22024.12 | 76.5 | — | — | |
| cViViT-Lcausality=causal, Pre-Train=IN-21K, Param (M)=310.92024.12 | 76.3 | — | — | |
| I3Dcausality=non-causal, Pre-Train=IN-1K, Param (M)=25.02024.12 | 72.1 | — | — | |
| EVA-02-CLIPZero-shot=true, Backbone=ViT-E/14+, Training Scale=Enlarged2023.03 | 69.8 | — | — | |
| EVA-02-CLIPZero-shot=true, Backbone=ViT-E/14, Training Scale=Enlarged2023.03 | 68.6 | — | — | |
| EVA-01-CLIPZero-shot=true, Backbone=ViT-g/14+, Training Scale=Enlarged2023.03 | 66.7 | — | — | |
| EVA-02-CLIPZero-shot=true, Backbone=ViT-L/14+, Training Scale=Enlarged2023.03 | 65.9 | — | — | |
| Open CLIPZero-shot=true, Backbone=ViT-G/14, Training Scale=Enlarged2023.03 | 65.9 | — | — | |
| EVA-02-CLIPZero-shot=true, Backbone=ViT-L/14, Training Scale=Standard2023.03 | 65 | — | — | |
| OpenAI CLIPZero-shot=true, Backbone=ViT-L/14+, Training Scale=Enlarged2023.03 | 64.9 | — | — | |
| OpenAI CLIPZero-shot=true, Backbone=ViT-L/14, Training Scale=Standard2023.03 | 64.5 | — | — | |
| Open CLIPZero-shot=true, Backbone=ViT-g/14, Training Scale=Enlarged2023.03 | 63.9 | — | — | |
| EVA-01-CLIPZero-shot=true, Backbone=ViT-g/14, Training Scale=Enlarged2023.03 | 63.2 | — | — | |
| Open CLIPZero-shot=true, Backbone=ViT-H/14, Training Scale=Enlarged2023.03 | 63.1 | — | — | |
| VFCZero-shot=true, Number of parameters=164M2023.04 | 58.8 | 84.5 | 71.7 | |
| Open CLIPZero-shot=true, Backbone=ViT-L/14, Training Scale=Standard2023.03 | 58 | — | — | |
| OpenAI CLIPZero-shot=true, Backbone=ViT-B/16, Training Scale=Standard2023.03 | 57.6 | — | — | |
| EVA-02-CLIPZero-shot=true, Backbone=ViT-B/16, Training Scale=Standard2023.03 | 57.4 | — | — | |
| Open CLIPZero-shot=true, Backbone=ViT-B/16, Training Scale=Standard2023.03 | 54.5 | — | — | |
| Flamingo-9BZero-shot=true, Number of parameters=9B2023.04 | 49.7 | 71.5 | 60.6 | |
| Flamingo-80BZero-shot=true, Number of parameters=80B2023.04 | 49.1 | 71.5 | 60.3 | |
| CLIPZero-shot=true, Number of parameters=151M2023.04 | 47.9 | 75.1 | 61.5 | |
| Flamingo-3BZero-shot=true, Number of parameters=3B2023.04 | 45.2 | 66.8 | 56 |