Action Recognition on Something-Something v1 (val)
67.3Top-1 AccSide4Video
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Side4VideoEvaluation Protocol=Frozen backbone, Backbones=ViT-E/14, Pre-train=LAION-2B, Frames × Views=16×3×2, TFLOPS=15.96×62023.11 | 67.3 | 88.8 | — | |
| ATMEvaluation Protocol=Full Fine-tuning, Backbones=ViT-L/14, Pre-train=Merged-2B, Frames × Views=16×3×2, TFLOPS=0.84×62023.11 | 65.6 | 88.6 | — | |
| Side4VideoEvaluation Protocol=Frozen backbone, Backbones=ViT-E/14, Pre-train=LAION-2B, Frames × Views=8×3×2, TFLOPS=7.98×62023.11 | 65.3 | 88.5 | — | |
| TDS-CLIP-L/14Training Protocol=Frozen CLIP, Pre-train=CLIP-400M, Frames × Cr. × Cl.=16×3×2, TFLOPS=1.6×62024.08 | 64.1 | 88.2 | — | |
| ATMEvaluation Protocol=Full Fine-tuning, Backbones=ViT-L/14, Pre-train=CLIP-400M, Frames × Views=16×3×2, TFLOPS=0.84×62023.11 | 64 | 88 | — | |
| TDS-CLIP-L/14Training Protocol=Frozen CLIP, Pre-train=CLIP-400M, Frames × Cr. × Cl.=8×3×2, TFLOPS=0.8×62024.08 | 63 | 87.8 | — | |
| UniFormerV2Evaluation Protocol=Full Fine-tuning, Backbones=ViT-L/14, Pre-train=CLIP-400M, Frames × Views=32×3×1, TFLOPS=1.73×32023.11 | 62.7 | 88 | — | |
| Side4VideoEvaluation Protocol=Frozen backbone, Backbones=ViT-L/14, Pre-train=CLIP-400M, Frames × Views=16×3×2, TFLOPS=1.74×62023.11 | 62.4 | 88.1 | — | |
| Side4VideoEvaluation Protocol=Frozen backbone, Backbones=ViT-L/14, Pre-train=CLIP-400M, Frames × Views=8×3×2, TFLOPS=0.87×62023.11 | 61 | 86.7 | — | |
| TDS-CLIP-B/16Training Protocol=Frozen CLIP, Pre-train=CLIP-400M, Frames × Cr. × Cl.=16×3×2, TFLOPS=0.4×62024.08 | 61 | 86.3 | — | |
| Side4VideoEvaluation Protocol=Frozen backbone, Backbones=ViT-B/16, Pre-train=CLIP-400M, Frames × Views=16×3×2, TFLOPS=0.36×62023.11 | 60.7 | 86 | — | |
| Side4Video-B/16Training Protocol=Frozen CLIP, Pre-train=CLIP-400M, Frames × Cr. × Cl.=16×3×2, TFLOPS=0.4×62024.08 | 60.7 | 86 | — | |
| UniFormer V2-L/14Training Protocol=Full Fine-tuning, Pre-train=CLIP-400M, Frames × Cr. × Cl.=16×3×1, TFLOPS=0.9×32024.08 | 60.5 | 86.5 | — | |
| TDS-CLIP-B/16Training Protocol=Frozen CLIP, Pre-train=CLIP-400M, Frames × Cr. × Cl.=8×3×2, TFLOPS=0.2×62024.08 | 60.1 | 85.4 | — | |
| Side4VideoEvaluation Protocol=Frozen backbone, Backbones=ViT-B/16, Pre-train=CLIP-400M, Frames × Views=8×3×2, TFLOPS=0.18×62023.11 | 59.4 | 84.8 | — | |
| Uniformer-FFNArchitecture=Uniformer, Number of Frames=162023.03 | 58.88 | — | — | |
| MorphMLP-BPretrain=IN-1K, #Frame=32×3×1, GFLOPS=5912021.11 | 57.4 | 84.5 | — | |
| EANEn(RGB+LMC)Backbone=ResNet-50, Pre-train=ImageNet, Frames=(8+16)×5, GFLOPs=1112021.07 | 57.2 | 83.9 | — | |
| UniformerArchitecture=Uniformer, Number of Frames=162023.03 | 56.71 | — | — | |
| Uniformer-STArchitecture=Uniformer, Number of Frames=162023.03 | 56.71 | — | — | |
| Uniformer-FFNArchitecture=Uniformer, Number of Frames=82023.03 | 56.64 | — | — | |
| GC-TDNParams=27.4M x 2, Frames=8+16, FLOPs x Clips=110.1G x 12022.03 | 56.4 | 84 | — | |
| EANEn(RGB)Backbone=ResNet-50, Pre-train=ImageNet, Frames=8+16, GFLOPs=1082021.07 | 55.8 | 83.1 | — | |
| MorphMLP-BPretrain=IN-1K, #Frame=16×3×1, GFLOPS=2942021.11 | 55.5 | 82.4 | — | |
| PAN_EnBackbone=ResNet-101+TSM, Flow=false, #Frame=(8+8x4)x2, FLOPs x views=(85.6G+166.1G)x22020.08 | 55.3 | 82.8 | — | |
| GC-TSMParams=25.1M x 2, Frames=8+16, FLOPs x Clips=99.8G x 62022.03 | 55.3 | 82.7 | — | |
| PANBackbone=ResNet-101, Pre-train=ImageNet, Frames=(8×5)×2, GFLOPs=5032021.07 | 55.3 | 82.8 | — | |
| TDNParams=26.1M x 2, Frames=8+16, FLOPs x Clips=108.0G x 12022.03 | 55.1 | 82.9 | — | |
| TDNEn(RGB+SDM)Backbone=ResNet-50, Pre-train=ImageNet, Frames=(8+16)×5, GFLOPs=1082021.07 | 55.1 | 82.9 | — | |
| SSAEnBackbone=ResNet-50, Modality=RGB, Frames=16+82021.05 | 55.1 | 84.9 | — | |
| GC-TSMParams=25.1M x 2, Frames=8+16, FLOPs x Clips=99.8G x 22022.03 | 55 | 82.6 | — | |
| GC-TDNParams=27.4M, Frames=16, FLOPs x Clips=73.4G x 12022.03 | 55 | 82.3 | — | |
| EAN16F(RGB+LMC)Backbone=ResNet-50, Pre-train=ImageNet, Frames=16×5, GFLOPs=742021.07 | 54.7 | 82.3 | — | |
| TEA-FFNArchitecture=TEA, Number of Frames=162023.03 | 54.04 | — | — | |
| MVFNetBackbone=ResNet50, Frames × Crops × Clips=(16+8)×3×2, FLOPs=99G×3×2, Pre-train=ImageNet, Ensemble=true2020.12 | 54 | — | — | |
| TDNParams=26.1M, Frames=16, FLOPs x Clips=72.0G x 12022.03 | 53.9 | 82.1 | — | |
| TDN16F(RGB+SDM)Backbone=ResNet-50, Pre-train=ImageNet, Frames=16×5, GFLOPs=722021.07 | 53.9 | 82.1 | — | |
| MorphMLP-SPretrain=IN-1K, #Frame=16×3×1, GFLOPS=2012021.11 | 53.9 | 81.3 | — | |
| Collaborative Memory (SlowFast-101+NL 8×8)Backbone=SlowFast-101+NL 8×8, Pretrain=K400, Only RGB=true2021.04 | 53.7 | — | — | |
| GC-TSNParams=25.1M x 2, Frames=8+16, FLOPs x Clips=99.8G x 22022.03 | 53.7 | 81.8 | — | |
| GC-TDNParams=27.4M, Frames=8, FLOPs x Clips=36.7G x 12022.03 | 53.7 | 82.2 | — | |
| PAN_EnBackbone=ResNet-50+TSM, Flow=false, #Frame=(8+8x4)x2, FLOPs x views=(46.6G+88.4G)x22020.08 | 53.4 | 81.1 | — | |
| EAN16F(RGB)Backbone=ResNet-50, Pre-train=ImageNet, Frames=16, GFLOPs=722021.07 | 53.4 | 81.4 | — | |
| EAN8F(RGB+LMC)Backbone=ResNet-50, Pre-train=ImageNet, Frames=8×5, GFLOPs=372021.07 | 53.4 | 81.1 | — | |
| CT-NETPretrain=IN-1K, #Frame=16×3×2, GFLOPS=4472021.11 | 53.4 | 81.7 | — | |
| CorrNet-101Pretrain=Sports1M, Only RGB=true2021.04 | 53.3 | — | — | |
| bLVNet-TAMBackbone=bLResNet-101, Pretrain=SS-V1, Frames=32×2, Modality=RGB, Param (10^6)=40.2, FLOPs (10^9)=128.62019.12 | 53.1 | 82.9 | — | |
| GC-TSMParams=25.1M, Frames=16, FLOPs x Clips=66.5G x 22022.03 | 53.1 | 81.2 | — | |
| MARS+RGB+FlowPretrain=K400, Only RGB=false2021.04 | 53 | — | — | |
| RNL-TSMParams=35.5M x 2, Frames=8+16, FLOPs x Clips=123.5G x 22022.03 | 52.7 | — | — | |
| TSM RGB+FlowBackbone=ResNet-50, Flow=true, #Frame=16+16x62020.08 | 52.6 | 81.9 | — | |
| MVFNetBackbone=ResNet50, Frames × Crops × Clips=16×3×2, FLOPs=66G×3×2, Pre-train=ImageNet2020.12 | 52.6 | — | — | |
| Two-stream TSMPretrain=ImageNet, Only RGB=false2021.04 | 52.6 | — | — | |
| TSMBackbone=ResNet-50, Pretrain=Kinetics, Frames=16+16, Modality=RGB+Flow2019.12 | 52.6 | 81.9 | — | |
| TSMRGB+FlowBackbone=ResNet-50, Pre-train=ImageNet, Frames=16×72021.07 | 52.6 | 81.9 | — | |
| TSM 2-streamBackbone=ResNet-50, Modality=RGB+flow, Frames=16+162021.05 | 52.6 | 81.9 | — | |
| MUSLEBackbone=ResNet-50, Pre-train=ImageNet, #Frames=162022.01 | 52.5 | 81.6 | — | |
| TEINetParams=30.4M x 2, Frames=8+16, FLOPs x Clips=99.0G x 12022.03 | 52.5 | — | — | |
| GC-GSTParams=22.3M x 2, Frames=8+16, FLOPs x Clips=88.7G x 22022.03 | 52.5 | 81.3 | — | |
| TEINetEnBackbone=ResNet-50, Pre-train=ImageNet, Frames=8+16, GFLOPs=992021.07 | 52.5 | — | — | |
| TEA_16FBackbone=ResNet-50, Flow=false, #Frame=16, FLOPs x views=70Gx302020.08 | 52.3 | 81.9 | — | |
| TEABackbone=ResNet50, Frames × Crops × Clips=16×3×10, FLOPs=70G×3×10, Pre-train=ImageNet2020.12 | 52.3 | — | — | |
| Collaborative Memory (SlowFast-50 8×8)Backbone=SlowFast-50 8×8, Pretrain=K400, Only RGB=true2021.04 | 52.3 | — | — | |
| TEAFrames=16, FLOPs x Clips=70.0G x 302022.03 | 52.3 | 81.9 | — | |
| TEABackbone=ResNet-50, Pre-train=ImageNet, Frames=16×30, GFLOPs=21002021.07 | 52.3 | 81.9 | — | |
| TDN8F(RGB+SDM)Backbone=ResNet-50, Pre-train=ImageNet, Frames=8×5, GFLOPs=362021.07 | 52.3 | 80.6 | — | |
| TEAPretrain=IN-1K, #Frame=16×3×10, GFLOPS=21002021.11 | 52.3 | 81.9 | — | |
| bLVNet-TAMBackbone=bLResNet-101, Pretrain=SS-V1, Frames=24×2, Modality=RGB, Param (10^6)=40.2, FLOPs (10^9)=96.42019.12 | 52.2 | 81.8 | — | |
| MSNetPretrain=ImageNet, Only RGB=true2021.04 | 52.1 | — | — | |
| TEABackbone=ResNet-50, Pre-train=ImageNet, #Frames=162022.01 | 51.9 | 80.3 | — | |
| TEA16FBackbone=ResNet-50, Pre-train=ImageNet, Frames=16, GFLOPs=702021.07 | 51.9 | 80.3 | — | |
| EAN8F(RGB)Backbone=ResNet-50, Pre-train=ImageNet, Frames=8, GFLOPs=362021.07 | 51.9 | 79.5 | — | |
| DSA+I3DBackbone=3D ResNet502021.05 | 51.8 | — | — | |
| TEA_8FBackbone=ResNet-50, Flow=false, #Frame=8, FLOPs x views=35Gx302020.08 | 51.7 | 80.5 | — | |
| TEABackbone=ResNet50, Frames × Crops × Clips=8×3×10, FLOPs=35G×3×10, Pre-train=ImageNet2020.12 | 51.7 | — | — | |
| SSABackbone=ResNet-50, Modality=RGB, Frames=162021.05 | 51.7 | 81.3 | — | |
| TEA-FFNArchitecture=TEA, Number of Frames=82023.03 | 51.61 | — | — | |
| Uniformer-STArchitecture=Uniformer, Number of Frames=82023.03 | 51.49 | — | — | |
| Uniformer-FFNArchitecture=Uniformer, Number of Frames=42023.03 | 51.41 | — | — | |
| ASSBackbone=ResNet-50, Pre-train=ImageNet, #Frames=322022.01 | 51.4 | — | — | |
| GC-TSNParams=25.1M, Frames=16, FLOPs x Clips=66.5G x 22022.03 | 51.3 | 80 | — | |
| TEAArchitecture=TEA, Number of Frames=162023.03 | 51.23 | — | — | |
| TEA-STArchitecture=TEA, Number of Frames=162023.03 | 51.23 | — | — | |
| SlowFast-101+NL 8×8Pretrain=K400, Only RGB=true, reproduced_by_authors=true2021.04 | 51.2 | — | — | |
| GC-TSMParams=25.1M, Frames=8, FLOPs x Clips=33.3G x 22022.03 | 51.1 | 79.4 | — | |
| TEINetBackbone=ResNet50, Frames × Crops × Clips=16×3×10, FLOPs=66G×3×10, Pre-train=ImageNet2020.12 | 51 | — | — | |
| MVFNetBackbone=ResNet50, Frames × Crops × Clips=16×1×1, FLOPs=66G×1×1, Pre-train=ImageNet2020.12 | 51 | — | — | |
| TEINetPretrain=IN-1K, #Frame=16×3×10, GFLOPS=19802021.11 | 51 | — | — | |
| TSM-FFNArchitecture=TSM, Number of Frames=162023.03 | 50.79 | — | — | |
| STMBackbone=ResNet-50, Pretrain=ImageNet, Frame=162019.08 | 50.7 | 80.4 | — | |
| STMBackbone=ResNet50, Frames × Crops × Clips=16×3×10, FLOPs=67G×3×10, Pre-train=ImageNet2020.12 | 50.7 | — | — | |
| STMBackbone=ResNet-50, Pre-train=ImageNet, #Frames=162022.01 | 50.7 | 80.4 | — | |
| STMParams=24.0M, Frames=16, FLOPs x Clips=33.3G x 302022.03 | 50.7 | 80.4 | — | |
| STMBackbone=ResNet-50, Pre-train=ImageNet, Frames=16×30, GFLOPs=20102021.07 | 50.7 | 80.4 | — | |
| GSMBackbone=InceptionV3, Pre-train=ImageNet, #Frames=162022.01 | 50.6 | — | — | |
| TANetParams=25.1M x 2, Frames=8+16, FLOPs x Clips=99.0G x 12022.03 | 50.6 | 79.3 | — | |
| TANetEnBackbone=ResNet-50, Pre-train=ImageNet, Frames=8+16, GFLOPs=992021.07 | 50.6 | 79.3 | — | |
| MorphMLP-SPretrain=IN-1K, #Frame=16×1×1, GFLOPS=672021.11 | 50.6 | 78 | — | |
| PAN_FullBackbone=ResNet-50+TSM, Flow=false, #Frame=8+8x4, FLOPs x views=67.7Gx12020.08 | 50.5 | 79.2 | — | |
| MVFNetBackbone=ResNet50, Frames × Crops × Clips=8×3×2, FLOPs=33G×3×2, Pre-train=ImageNet2020.12 | 50.5 | — | — |