Action Recognition on HMDB51 (Top-1 Accuracy)
89.3Top-1 AccInternVideo
Evaluation results
| Method | Links | |
|---|---|---|
| InternVideo2022.12 | 89.3 | |
| MTV2022.12 | 87.6 | |
| R(2+1)D BERT (64f)Uses Flow?=No, Extra Training Data=IG65M2020.08 | 85.1 | |
| R(2+1)D BERT (32f)Uses Flow?=No, Extra Training Data=IG65M2020.08 | 83.99 | |
| ResNeXt101 BERTUses Flow?=Yes, Extra Training Data=Kinetics-4002020.08 | 83.55 | |
| BIKE (ViT-L, 336)Pre-training=Kinetics, Modality=RGB, Backbone=ViT-L, Resolution=3362022.12 | 83.1 | |
| HAF + BoW/FV hallucUses Flow?=No, Extra Training Data=Kinetics-4002020.08 | 82.48 | |
| EvaNetUses Flow?=Yes, Extra Training Data=Kinetics-4002020.08 | 82.3 | |
| BIKE (ViT-L)Pre-training=Kinetics, Modality=RGB, Backbone=ViT-L2022.12 | 82.2 | |
| ResNeXt101Uses Flow?=Yes, Extra Training Data=Kinetics-4002020.08 | 81.78 | |
| FcFUses Flow?=No, Extra Training Data=Kinetics-4002020.08 | 81.1 | |
| I3DUses Flow?=Yes, Extra Training Data=Kinetics-4002020.08 | 80.9 | |
| MARS + RGB + FlowUses Flow?=Yes, Extra Training Data=Kinetics-4002020.08 | 80.9 | |
| Two-stream I3DBackbone=3D Inception-v1, Pre-train=Kinetics, #3D=true, #Optical flow=true, #Frames=64+642021.03 | 80.9 | |
| InternVideo2-s2-6BSetting=16 x 224, Protocol=Linear probing2024.03 | 80.7 | |
| R(2+1)D (32f)Uses Flow?=No, Extra Training Data=IG65M2020.08 | 80.54 | |
| RSTAN + IDTUses Flow?=Yes, Extra Training Data=ImageNet2020.08 | 79.9 | |
| R(2+1)DUses Flow?=Yes, Extra Training Data=Kinetics-4002020.08 | 78.7 | |
| TDNPre-training=Kinetics, Modality=RGB2022.12 | 76.4 | |
| S3D-GPre-training=Kinetics, Modality=RGB2022.12 | 75.9 | |
| MSNet-R50Backbone=TSM-ResNet50, Pre-train=Kinetics, #Frames=82021.03 | 75.8 | |
| MVFNetPre-training=Kinetics, Modality=RGB2022.12 | 75.7 | |
| MMV (Audio, Text)Modality=Video + Audio + Text, Pre-training video length (kilo hours)=139.8, Evaluation Protocol=Fully fine-tuning2022.01 | 75 | |
| MMVModality=A, T, Length (K)=139.8, Evaluation Protocol=Full2022.04 | 75 | |
| I3DPre-training=Kinetics, Modality=RGB2022.12 | 74.8 | |
| R(2+1)DUses Flow?=No, Extra Training Data=Kinetics-4002020.08 | 74.5 | |
| R(2+1)DPre-training=Kinetics, Modality=RGB2022.12 | 74.5 | |
| RGB I3DBackbone=3D Inception-v1, Pre-train=Kinetics, #3D=true, #Frames=642021.03 | 74.3 | |
| TSMUses Flow?=No, Extra Training Data=Kinetics-4002020.08 | 73.5 | |
| TSMBackbone=ResNet50, Pre-train=Kinetics, #Frames=82021.03 | 73.5 | |
| TSMPre-training=Kinetics, Modality=RGB2022.12 | 73.5 | |
| Ours-cumulativeBackbone=TSM-ResNet50, Pre-train=Kinetics, #Frames=all, #Clusters=82021.03 | 73.4 | |
| Ours-slopeBackbone=TSM-ResNet50, Pre-train=Kinetics, #Frames=all, #Clusters=82021.03 | 73.3 | |
| ECO-En LiteBackbone=BNIncep+3D Res18, Pre-train=Kinetics, #3D=true, #Frames=82021.03 | 72.4 | |
| STMBackbone=ResNet50, Pre-train=ImageNet+Kinetics, #Frames=162021.03 | 72.2 | |
| STMPre-training=Kinetics, Modality=RGB2022.12 | 72.2 | |
| InternVideo2-s1-6BSetting=16 x 224, Protocol=Linear probing2024.03 | 71.8 | |
| InternVideo2-s1-1BSetting=16 x 224, Protocol=Linear probing2024.03 | 71.6 | |
| SI+DI+OF+DOFBackbone=ResNeXt50, Pre-train=Imagenet, #Optical flow=dynamic images2021.03 | 71.5 | |
| IMD-B2022.03 | 71.3 | |
| TSNUses Flow?=Yes, Extra Training Data=ImageNet2020.08 | 71 | |
| ARTNetPre-training=Kinetics, Modality=RGB2022.12 | 70.9 | |
| MILESModality=T, Length (K)=13.0, Evaluation Protocol=Full2022.04 | 70.5 | |
| COVIARModality=RGB + Optical Flow2022.03 | 70.2 | |
| ActionVlad + IDTUses Flow?=Yes, Extra Training Data=ImageNet2020.08 | 69.8 | |
| BridgeFormerModality=Video + Text, Pre-training video length (kilo hours)=13.0, Evaluation Protocol=Fully fine-tuning2022.01 | 69.8 | |
| TSN2022.03 | 69.4 | |
| Two-stream Fusion + IDTUses Flow?=Yes, Extra Training Data=ImageNet2020.08 | 69.2 | |
| XDCArchitecture=R(2+1)D-18, Pretraining Dataset=IG-Kinetics2019.11 | 68.9 | |
| ELoArchitecture=R(2+1)D-50, Pretraining Dataset=YouTube-8M2019.11 | 67.4 | |
| ELOModality=Video + Audio + Optical Flow, Pre-training video length (kilo hours)=115.0, Evaluation Protocol=Fully fine-tuning2022.01 | 67.4 | |
| ELOModality=A, OF, Length (K)=115.0, Evaluation Protocol=Full2022.04 | 67.4 | |
| MMV (Audio, Text)Modality=Video + Audio + Text, Pre-training video length (kilo hours)=139.8, Evaluation Protocol=Linear2022.01 | 67.1 | |
| MMVModality=A, T, Length (K)=139.8, Evaluation Protocol=Linear2022.04 | 67.1 | |
| XDCArchitecture=R(2+1)D-18, Pretraining Dataset=IG-Random2019.11 | 66.5 | |
| Two-Stream I3DNumber of Parameters=25M2021.04 | 66.4 | |
| I3D2022.03 | 66.4 | |
| FrozenModality=Video + Text, Pre-training video length (kilo hours)=13.0, Evaluation Protocol=Fully fine-tuning2022.01 | 66.3 | |
| FrozenModality=T, Length (K)=13.0, Evaluation Protocol=Full2022.04 | 66.3 | |
| BridgeFormerModality=Video + Text, Pre-training video length (kilo hours)=13.0, Evaluation Protocol=Linear2022.01 | 65.8 | |
| TVTSv2-HSetting=12 x 224, Protocol=Linear probing2024.03 | 65.7 | |
| Two-stream fusion2022.03 | 65.4 | |
| MILESModality=T, Length (K)=13.0, Evaluation Protocol=Linear2022.04 | 65.4 | |
| Fully supervisedArchitecture=R(2+1)D-18, Pretraining Dataset=Kinetics2019.11 | 65.1 | |
| TSMBackbone=ResNet50, Pre-train=Kinetics, Implementation=our impl., #Frames=12021.03 | 65.1 | |
| TSNBackbone=ResNet50, Pre-train=Kinetics, #Frames=82021.03 | 64.7 | |
| ELOModality=Video + Audio + Optical Flow, Pre-training video length (kilo hours)=115.0, Evaluation Protocol=Linear2022.01 | 64.5 | |
| ELOModality=A, OF, Length (K)=115.0, Evaluation Protocol=Linear2022.04 | 64.5 | |
| CNN + HT-LSTMNumber of Parameters=22M2021.04 | 64.2 | |
| CNN + FDHT-LSTMNumber of Parameters=22M2021.04 | 64.2 | |
| CNN + TR-LSTMNumber of Parameters=39M2021.04 | 63.8 | |
| XDCArchitecture=R(2+1)D-18, Pretraining Dataset=AudioSet2019.11 | 63.7 | |
| MVCGCModality=Video + Motion Vector, Pre-training video length (kilo hours)=1.8, Evaluation Protocol=Fully fine-tuning2022.01 | 63.4 | |
| MVCGCModality=MV, Length (K)=1.8, Evaluation Protocol=Full2022.04 | 63.4 | |
| VATTModality=Video + Audio + Text, Pre-training video length (kilo hours)=139.8, Evaluation Protocol=Linear2022.01 | 63.3 | |
| VATTModality=A, T, Length (K)=139.8, Evaluation Protocol=Linear2022.04 | 63.3 | |
| TDD + FVNumber of Parameters=117M2021.04 | 63.2 | |
| XDC_KModality=Video + Audio, Pre-training video length (kilo hours)=188.3, Evaluation Protocol=Fully fine-tuning2022.01 | 63.1 | |
| XDC_KModality=A, Length (K)=188.3, Evaluation Protocol=Full2022.04 | 63.1 | |
| CNN + LSTMNumber of Parameters=55M2021.04 | 62.9 | |
| COCLRModality=Video + Optical Flow, Pre-training video length (kilo hours)=1.8, Evaluation Protocol=Fully fine-tuning2022.01 | 62.9 | |
| COCLRModality=OF, Length (K)=1.8, Evaluation Protocol=Full2022.04 | 62.9 | |
| VGG + Two-Stream FusionNumber of Parameters=181M2021.04 | 62.1 | |
| IDTUses Flow?=Yes2020.08 | 61.7 | |
| CtPPre-training Dataset=K400, Architecture=R(2+1)D2021.05 | 61.7 | |
| FrozenModality=Video + Text, Pre-training video length (kilo hours)=13.0, Evaluation Protocol=Linear2022.01 | 61.3 | |
| FrozenModality=T, Length (K)=13.0, Evaluation Protocol=Linear2022.04 | 61.3 | |
| XDC_RModality=Video + Audio, Pre-training video length (kilo hours)=188.3, Evaluation Protocol=Fully fine-tuning2022.01 | 61.2 | |
| XDC_RModality=A, Length (K)=188.3, Evaluation Protocol=Full2022.04 | 61.2 | |
| MIL-NCEArchitecture=S3D, Pretraining Dataset=HowTo100M2019.11 | 61 | |
| Two-StreamUses Flow?=Yes, Extra Training Data=ImageNet2020.08 | 59.4 | |
| MIL-NCEModality=Video + Text, Pre-training video length (kilo hours)=134.5, Evaluation Protocol=Fully fine-tuning2022.01 | 59.2 | |
| MIL-NCEModality=T, Length (K)=134.5, Evaluation Protocol=Full2022.04 | 59.2 | |
| COVIARModality=Compressed Domain2022.03 | 59.1 | |
| IMP-MoE-LPPT=350M, TPU-DAYS=1.5k, Zero-shot=true2023.05 | 59.1 | |
| VideoCoCaPPT=2B, TPU-DAYS=10.5k, Zero-shot=true2023.05 | 58.6 | |
| AVTS+Architecture=R(2+1)D-18, Pretraining Dataset=AudioSet, Evaluation Protocol=10 uniformly-sampled clips2019.11 | 58.1 | |
| AVTS*Architecture=MC3-18, Pretraining Dataset=AudioSet, Evaluation Protocol=Without dense prediction (10 clips)2019.11 | 57.3 | |
| VIFI-CLIPMode=Zero-shot, VideoPrompter=true2023.10 | 57.12 | |
| CtPPre-training Dataset=UCF, Architecture=R(2+1)D2021.05 | 57.1 |