Natural Language Video Localization on Charades-STA (test)
63.03R@1 (IoU=0.5)Three-stream DRFT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Three-stream DRFTModality=RGB + Flow + Depth2021.07 | 63.03 | 76.68 | 40.15 | 54.89 | |
| Two-stream DRFTModality=RGB + Flow2021.07 | 61.93 | 74.26 | 38.69 | 53.92 | |
| Single-stream DRFTModality=RGB2021.07 | 60.79 | 73.85 | 36.72 | 52.64 | |
| LGIModality=RGB2021.07 | 59.46 | 72.96 | 35.48 | 51.38 | |
| UniVTGBlock=Transformer (4-layer)2024.03 | 58.01 | 70.81 | 35.65 | — | |
| UniVTGBlock=Transformer (6-layer, reproduced)2024.03 | 57.26 | 68.2 | 34.68 | — | |
| UniVTG DBMBlock=DBM2024.03 | 57.18 | 68.06 | 36.05 | — | |
| UniVTGBlock=ViM2024.03 | 57.07 | 68.12 | 35.83 | — | |
| VSLNetFine-tuning=Yes, Visual Feature Extractor=3D ConvNet2020.04 | 54.19 | 70.46 | 35.22 | 50.02 | |
| BSPPre-training Dataset=Kinetics-400, Pre-training Supervision=Fully-supervised, Evaluation Framework=LGI2022.03 | 53.6 | 68.8 | 29.3 | 50.6 | |
| DRNModality=RGB2021.07 | 53.09 | — | 31.75 | — | |
| MDETRBlock=-2024.03 | 52.07 | 65.83 | 30.59 | — | |
| PfTML-GAModality=RGB2021.07 | 52.02 | 67.53 | 33.74 | — | |
| IVG-DCLType=Regression2021.06 | 50.24 | 67.63 | 32.88 | 48.02 | |
| VSLBaseFine-tuning=Yes, Visual Feature Extractor=3D ConvNet2020.04 | 50.23 | 68.06 | 30.16 | 47.15 | |
| PALPre-training Dataset=Kinetics-400, Pre-training Supervision=Unsupervised, Evaluation Framework=LGI2022.03 | 50 | 63.7 | 27.2 | 46.8 | |
| VSLNetFine-tuning=No, Visual Feature Extractor=3D ConvNet2020.04 | 47.31 | 64.3 | 30.19 | 45.15 | |
| VSLNetType=Regression2021.06 | 47.31 | 64.3 | 30.19 | 45.15 | |
| TACPre-training Dataset=Kinetics-400, Pre-training Supervision=Fully-supervised, Implementation=Authors' implementation, Evaluation Framework=LGI2022.03 | 46.8 | 61.6 | 24.6 | 44.3 | |
| MANType=Ranking2021.06 | 46.53 | — | 22.72 | — | |
| MAN2021.07 | 46.53 | — | 22.72 | — | |
| DRNType=Regression2021.06 | 45.4 | — | 26.4 | — | |
| VideoMoCoPre-training Dataset=Kinetics-400, Pre-training Supervision=Unsupervised, Evaluation Framework=LGI2022.03 | 44.5 | 59.1 | 23.4 | 42.3 | |
| ExCLFine-tuning=Yes, Visual Feature Extractor=3D ConvNet2020.04 | 44.1 | 65.1 | 23.3 | — | |
| RSPNetPre-training Dataset=Kinetics-400, Pre-training Supervision=Unsupervised, Evaluation Framework=LGI2022.03 | 41.5 | 55.8 | 21.4 | 39.6 | |
| VSLBaseFine-tuning=No, Visual Feature Extractor=3D ConvNet2020.04 | 40.97 | 61.72 | 24.14 | 42.11 | |
| VSLBaseType=Regression2021.06 | 40.97 | 61.72 | 24.14 | 42.11 | |
| MoCo-v2Pre-training Dataset=Kinetics-400, Pre-training Supervision=Unsupervised, Implementation=Authors' implementation, Evaluation Framework=LGI2022.03 | 40.9 | 54.2 | 21.1 | 38.7 | |
| 2D-TANType=Ranking2021.06 | 39.7 | — | 23.31 | — | |
| DEBUGFine-tuning=No, Visual Feature Extractor=3D ConvNet2020.04 | 37.39 | 54.95 | 17.69 | 36.34 | |
| DEBUGType=Regression2021.06 | 37.39 | 54.95 | 17.69 | 36.34 | |
| LT-ZVGArchitecture Type=Specialist model, # Train Samples=0, Setup=Zero-shot, Trained on raw videos (no annotations)=true2026.01 | 37.2 | 52.9 | 19.3 | 36 | |
| VIRTUE-Embed 7BArchitecture Type=MLLM-based model, # Train Samples=0, Setup=Zero-shot2026.01 | 36.8 | 55.5 | 16.6 | 35.9 | |
| RWMType=RL2021.06 | 36.7 | — | — | — | |
| RWM2021.07 | 36.7 | — | — | — | |
| TripNetModality=RGB2021.07 | 36.61 | 51.33 | 14.5 | — | |
| QSPNFine-tuning=No, Visual Feature Extractor=3D ConvNet2020.04 | 35.6 | 54.7 | 15.8 | — | |
| QSPNType=Ranking2021.06 | 35.6 | 54.7 | 15.8 | — | |
| VTimeLLMModel size=13B2023.11 | 34.3 | 55.3 | 14.7 | 34.6 | |
| ChatVTGArchitecture Type=MLLM-based model, # Train Samples=100K, Setup=Zero-shot2026.01 | 33 | 52.7 | 15.9 | 34.9 | |
| TimeChatArchitecture Type=MLLM-based model, # Train Samples=125K, Setup=Zero-shot2026.01 | 32.2 | — | 13.4 | — | |
| HawkEyeArchitecture Type=MLLM-based model, # Train Samples=715K, Setup=Zero-shot2026.01 | 31.4 | 50.6 | 14.5 | 33.7 | |
| MACType=Ranking2021.06 | 30.48 | — | 12.2 | — | |
| VTimeLLMModel size=7B2023.11 | 27.5 | 51 | 11.4 | 31.2 | |
| TimeLLMArchitecture Type=MLLM-based model, # Train Samples=170K, Setup=Zero-shot2026.01 | 27.5 | 51 | 11.4 | 31.2 | |
| SAPType=Ranking2021.06 | 27.42 | — | 13.36 | — | |
| MomenterArchitecture Type=MLLM-based model, # Train Samples=10M, Setup=Zero-shot2026.01 | 26.6 | 42.6 | 11.6 | 28.5 | |
| UniVTGArchitecture Type=Specialist model, # Train Samples=4.2M, Setup=Zero-shot2026.01 | 25.2 | 44.1 | 10 | 27.1 | |
| SM-RLType=RL2021.06 | 24.36 | — | 11.17 | — | |
| CTRLType=Ranking2021.06 | 23.63 | — | 8.89 | — | |
| MANFine-tuning=No, Visual Feature Extractor=3D ConvNet2020.04 | 22.72 | 46.53 | — | — | |
| CTRL2021.07 | 21.42 | — | 7.15 | — | |
| ACRNType=Ranking2021.06 | 20.26 | — | 7.64 | — | |
| VideoChat2Architecture Type=MLLM-based model, # Train Samples=2M, Setup=Zero-shot2026.01 | 14.3 | 38 | 3.8 | 24.6 | |
| SAPFine-tuning=No, Visual Feature Extractor=3D ConvNet2020.04 | 13.36 | 27.42 | — | — | |
| ACL-KFine-tuning=No, Visual Feature Extractor=3D ConvNet2020.04 | 12.2 | 30.48 | — | — | |
| SM-RLFine-tuning=No, Visual Feature Extractor=3D ConvNet2020.04 | 11.17 | 24.36 | — | — | |
| CTRLFine-tuning=No, Visual Feature Extractor=3D ConvNet2020.04 | 8.89 | 23.63 | — | — | |
| VideoChatGPTModel size=7B2023.11 | 7.7 | 20 | 1.7 | 13.7 | |
| VideoLLaMAModel size=7B2023.11 | 3.8 | 10.4 | 0.9 | 7.1 | |
| VideoChatModel size=7B2023.11 | 3.3 | 9 | 1.3 | 6.5 | |
| RWM-RLFine-tuning=No, Visual Feature Extractor=3D ConvNet2020.04 | — | 36.7 | — | — |