Action Recognition on UCF101 (mean of 3 splits)
98.7AccuracyPrevious SotA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Previous SotAModality=V2021.06 | 98.7 | — | — | |
| Two-Stream I3DPre-training=ImageNet+Kinetics2017.05 | 98 | — | — | |
| Two-stream I3DDim=3D2017.11 | 98 | — | — | |
| I3DVideo Input Type=Decoded, Input Modality=RGB + Flow, Backbone Architecture=I3D2019.01 | 98 | — | — | |
| I3D two-streamBackbone=3D Inception-v1, Flow=true, Pre-train Data=ImageNet+Kinetics2019.08 | 98 | — | — | |
| Two-Stream I3DPre-training=Kinetics2017.05 | 97.8 | — | — | |
| R(2+1)DVideo Input Type=Decoded, Input Modality=RGB + Flow2019.01 | 97.3 | — | — | |
| TSN two-StreamBackbone=BNInception, Flow=true, Pre-train Data=ImageNet+Kinetics2019.08 | 97 | — | — | |
| R(2+1)DInputs=RGB, Pre-train=Kinetics2017.12 | 96.8 | — | — | |
| S3D-GInputs=RGB, Pre-train=ImNet+Kinetics2017.12 | 96.8 | — | — | |
| S3DVideo Input Type=Decoded, Input Modality=RGB, Temporal Modeling Approach=Spatio-temporal modeling2019.01 | 96.8 | — | — | |
| Fully-supervised SOTABackbone=S3D, Frozen=false2019.12 | 96.8 | — | — | |
| S3D-GInput=64 x 3 x 224 x 224, Clips=ALL, Backbone=Inception2020.06 | 96.8 | — | — | |
| R(2+1)D-RGBInput=32 x 3 x 112 x 112, Clips=102020.06 | 96.8 | — | — | |
| R(2+1)D-RGB & DSNInput=32 x 3 x 112 x 112, Clips=4, M=12020.06 | 96.8 | — | — | |
| K400 Sup.Modality=V, Pre-Train Dataset=Kinetics-4002021.06 | 96.8 | — | — | |
| Flow-I3DPre-training=ImageNet+Kinetics2017.05 | 96.7 | — | — | |
| R(2+1)D-RGBInput=32 x 3 x 112 x 112, Clips=10, Backbone=ResNet-34, label=ours2020.06 | 96.7 | — | — | |
| Flow-I3DPre-training=Kinetics2017.05 | 96.5 | — | — | |
| I3D RGB + DMC-Net (I3D)Video Input Type=Decoded, Input Modality=RGB, Temporal Modeling Approach=Spatio-temporal modeling, Backbone Architecture=I3D + DMC-Net2019.01 | 96.5 | — | — | |
| K400 Self-Sup.Modality=V, Pre-Train Dataset=Kinetics-4002021.06 | 96.3 | — | — | |
| STMBackbone=ResNet-50, Flow=false, Pre-train Data=ImageNet+Kinetics2019.08 | 96.2 | — | — | |
| MF-NetVideo Input Type=Decoded, Input Modality=RGB, Temporal Modeling Approach=Spatio-temporal modeling2019.01 | 96 | — | — | |
| MF-NetInput=16 x 3 x 224 x 224, Clips=502020.06 | 96 | — | — | |
| DisentanglingBackbone=BNInception, Flow=false, Pre-train Data=ImageNet+Kinetics2019.08 | 95.9 | — | — | |
| RGB-I3DPre-train dataset=ImageNet+Kinetics, Spatial resolution=224 x 224, Backbone architecture=Inception V12017.11 | 95.6 | — | — | |
| I3DInputs=RGB, Pre-train=ImNet+Kinetics2017.12 | 95.6 | — | — | |
| RGB-I3DPre-training=ImageNet+Kinetics2017.05 | 95.6 | — | — | |
| I3D RGBVideo Input Type=Decoded, Input Modality=RGB, Temporal Modeling Approach=Spatio-temporal modeling, Backbone Architecture=I3D2019.01 | 95.6 | — | — | |
| I3D-RGBInput=64 x 3 x 224 x 224, Clips=ALL, Backbone=Inception V12020.06 | 95.6 | — | — | |
| MMVModality=V+A+T, Pre-Train Dataset=AS+HTM2021.06 | 95.2 | — | — | |
| GDTH/W x T=112 x 32, Backbone=R(2+1)D, Modality=V+A, Pretraining=IG65M, Evaluation Protocol=finetune2021.01 | 95.2 | — | — | |
| RGB-I3DPre-training=Kinetics2017.05 | 95.1 | — | — | |
| I3D RGBBackbone=3D Inception-v1, Flow=false, Pre-train Data=ImageNet+Kinetics2019.08 | 95.1 | — | — | |
| ECOBackbone=BNInception+3D ResNet-18, Flow=false, Pre-train Data=Kinetics2019.08 | 94.8 | — | — | |
| ST-ResNet+iDTBackbone Architecture=ResNet, Strategy=iDT2017.04 | 94.6 | — | — | |
| ST-ResNet + IDTPre-training=None2017.05 | 94.6 | — | — | |
| ResNeXt-101Dim=3D, Input=64f2017.11 | 94.5 | — | — | |
| TVNetVideo Input Type=Decoded, Input Modality=RGB, Temporal Modeling Approach=Motion representation learning2019.01 | 94.5 | — | — | |
| TSMBackbone=ResNet-50, Flow=false, Pre-train Data=ImageNet+Kinetics2019.08 | 94.5 | — | — | |
| ARTNet with TSNPre-train dataset=Kinetics, Spatial resolution=112 x 112, Backbone architecture=ResNet-182017.11 | 94.3 | — | — | |
| ARTNet w/ TSNInputs=RGB, Pre-train=Kinetics2017.12 | 94.3 | — | — | |
| ARTNetVideo Input Type=Decoded, Input Modality=RGB, Temporal Modeling Approach=Spatio-temporal modeling2019.01 | 94.3 | — | — | |
| ARTNet with TSNBackbone=3D ResNet-18, Flow=false, Pre-train Data=Kinetics2019.08 | 94.3 | — | — | |
| ARTNet with TSNInput=24 x 3 x 112 x 112, Clips=25, Backbone=ResNet-182020.06 | 94.3 | — | — | |
| TSNBackbone Architecture=BN-Inception, Modalities Used=3-modality2017.04 | 94.2 | — | — | |
| TSN2017.04 | 94.2 | — | — | |
| Temporal Segment NetworksPre-training=None2017.05 | 94.2 | — | — | |
| ST Multiplier NetDim=2D2017.11 | 94.2 | — | — | |
| TSNDim=2D2017.11 | 94.2 | — | — | |
| TSN(RGB+Flow)Speed (fps)=142017.11 | 94 | — | — | |
| STCBackbone=ResNet101, Flow=false, Pre-train Data=Kinetics2019.08 | 93.7 | — | — | |
| RSPNetBackbone=S3D-G, Input Size=64 x 224, Pre-training Dataset=Kinetics-4002021.01 | 93.7 | — | — | |
| ActionVLAD (VGG-16) + iDTBackbone Architecture=VGG-16, Strategy=iDT2017.04 | 93.6 | — | — | |
| ARTNet w/o TSNPre-train dataset=Kinetics, Spatial resolution=112 x 112, Backbone architecture=ResNet-182017.11 | 93.5 | — | — | |
| Two-Stream Fusion+iDTStrategy=iDT2017.04 | 93.5 | — | — | |
| Two-str. (conv. fusion)+IDTModality=RGB + Flow + IDT2016.04 | 93.5 | — | — | |
| Two-Stream Fusion + IDTPre-training=None2017.05 | 93.5 | — | — | |
| StNetBackbone=ResNet50, Flow=false, Pre-train Data=ImageNet+Kinetics2019.08 | 93.5 | — | — | |
| Two-Stream I3D (RGB+Flow)Speed (fps)=<142017.11 | 93.4 | — | — | |
| RGB+OFF(RGB)+RGB Diff+OFF(RGB Diff)Speed (fps)=2062017.11 | 93.3 | — | — | |
| TSN Spatial NetworkPre-train dataset=ImageNet+Kinetics, Spatial resolution=229 x 229, Backbone architecture=Inception V32017.11 | 93.2 | — | — | |
| TSN RGBInput=1 x 3 x 229 x 229, Clips=25, Backbone=Inception V32020.06 | 93.2 | — | — | |
| KVMF2017.04 | 93.1 | — | — | |
| KVMF2017.04 | 93.1 | — | — | |
| MoCoModality=V, Pre-Train Dataset=IG-Uncurated2021.06 | 92.9 | — | — | |
| ActionVLAD (LateFuse, VGG-16)Backbone Architecture=VGG-16, Fusion=LateFuse2017.04 | 92.7 | — | — | |
| LTC+iDTStrategy=iDT2017.04 | 92.7 | — | — | |
| LTC Flow+RGB+IDTModality=RGB + Flow + IDT2016.04 | 92.7 | — | — | |
| VIMPACModality=V, Pre-Train Dataset=HTM2021.06 | 92.7 | — | — | |
| Spatiotemporal fusion ConvNetSpatial stream backbone=VGG-16, Temporal stream backbone=VGG-16, Fusion strategy=3D Conv + 3D Pooling2016.04 | 92.5 | — | — | |
| Two-Stream FusionBackbone Architecture=VGG-162017.04 | 92.5 | — | — | |
| DT+Hybrid architecturesStrategy=Hybrid2017.04 | 92.5 | — | — | |
| TS Fusion2017.04 | 92.5 | — | — | |
| Two-stream (conv. fusion)Modality=RGB + Flow2016.04 | 92.5 | — | — | |
| Two-Stream fusionVideo Input Type=Decoded, Input Modality=RGB + Flow2019.01 | 92.5 | — | — | |
| Transformations2017.04 | 92.4 | — | — | |
| TransformationsModality=RGB + Flow2016.04 | 92.4 | — | — | |
| CPD2017.04 | 92.3 | — | — | |
| DMC-Net (I3D)Video Input Type=Compressed, Backbone Architecture=I3D2019.01 | 92.3 | — | — | |
| CVRLBackbone=R3D-50, Input Size=32 x 224, Pre-training Dataset=Kinetics-4002021.01 | 92.2 | — | — | |
| w/o TemporalModality=V2021.06 | 92 | — | — | |
| Spatiotemporal fusion ConvNetSpatial stream backbone=VGG-16, Temporal stream backbone=VGG-16, Fusion strategy=Single tower after fusion (3D Conv + 3D Pooling)2016.04 | 91.8 | — | — | |
| Two-Stream ConvNetSpatial stream backbone=VGG-16, Evaluation=Re-implementation (ours)2016.04 | 91.7 | — | — | |
| LTCBackbone Architecture=Long-term Temporal Conv2017.04 | 91.7 | — | — | |
| LTC2017.04 | 91.7 | — | — | |
| LTC Flow+RGBModality=RGB + Flow2016.04 | 91.7 | — | — | |
| TDD+IDTModality=RGB + Flow + IDT2016.04 | 91.5 | — | — | |
| TDD + IDTPre-training=None2017.05 | 91.5 | — | — | |
| Two-Stream ConvNetSpatial stream backbone=VGG-16, Evaluation=[35]2016.04 | 91.4 | — | — | |
| Two-Stream ConvNetBackbone Architecture=VGG-162017.04 | 91.4 | — | — | |
| MIL-NCEPre-training Dataset=HTM, Modality=Text, Backbone=S3D, Frozen=false2019.12 | 91.3 | — | — | |
| MIL-NCEModality=V+T, Pre-Train Dataset=HTM2021.06 | 91.3 | — | — | |
| MIL-NCEH/W x T=224 x -, Backbone=S3D, Modality=V+T, Pretraining=HowTo100M, Evaluation Protocol=finetune2021.01 | 91.3 | — | — | |
| TSN Spatial NetworkPre-train dataset=ImageNet+Kinetics, Spatial resolution=224 x 224, Backbone architecture=Inception V22017.11 | 91.1 | — | — | |
| Chained Multi-stream Networksmode=chained2017.04 | 91.1 | — | — | |
| TSN RGBBackbone=BNInception, Flow=false, Pre-train Data=ImageNet+Kinetics2019.08 | 91.1 | — | — | |
| TSN RGBInput=1 x 3 x 224 x 224, Clips=25, Backbone=Inception V22020.06 | 91.1 | — | — | |
| TSN(RGB+RGB Diff)Speed (fps)=3402017.11 | 91 | — | — | |
| DMC-Net (ResNet-18)Video Input Type=Compressed, Backbone Architecture=ResNet-182019.01 | 90.9 | — | — |