Video Captioning on MSRVTT (test)
80.7CIDErVideoLLaMA 2 + AVEX-Prune
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| VideoLLaMA 2 + AVEX-PruneParameter count=7B, retention ratio=40%2026.06 | 80.7 | 58.1 | 35.1 | — | |
| HierarQLLM Finetuning=true2025.03 | 80.5 | — | — | — | |
| VideoLLaMA 2Parameter count=7B2026.06 | 80.5 | 57.9 | 35 | — | |
| mPLUG-22024.11 | 80.3 | — | 34.9 | — | |
| mPLUG-22025.03 | 80.3 | — | — | — | |
| VILA 1.5 + AVEX-PruneParameter count=8B, retention ratio=40%2026.06 | 80 | 57.2 | 34 | — | |
| VILA 1.5Parameter count=8B2026.06 | 79.9 | 56.8 | 33.7 | — | |
| HierarQLLM Finetuning=false2025.03 | 79.8 | — | — | — | |
| LLaVA-NeXT-VideoParameter count=7B2026.06 | 78 | 56.3 | 33.1 | — | |
| GIT2Parameters=12.9B2023.06 | 75.9 | 54.8 | — | — | |
| GIT22022.05 | 75.9 | — | — | — | |
| COSA (1.2B)Parameters=415M2023.06 | 74.7 | 53.7 | — | — | |
| MA-LMM2024.11 | 74.6 | — | 33.4 | — | |
| MA-LMM2025.03 | 74.6 | — | — | — | |
| OmAgentParameter count=7B2026.06 | 74.1 | 52.9 | 31.5 | — | |
| VALOR-LParameters=433.5M, Additional modalities=true2023.06 | 74 | 54.4 | — | — | |
| GITParameters=1.7B2023.06 | 73.9 | 53.8 | — | — | |
| GIT2024.11 | 73.9 | — | 32.9 | — | |
| GIT2025.03 | 73.9 | — | — | — | |
| GIT2022.05 | 73.9 | — | — | — | |
| GIT2022.05 | 73.9 | 53.8 | — | — | |
| VideoCoca2024.11 | 73.2 | — | — | — | |
| AdaCM²2024.11 | 73.1 | — | 33 | — | |
| COSA-LParameters=417M2023.06 | 72.1 | 53.2 | — | — | |
| VideoLLaMA2024.11 | 71.6 | — | 32.9 | — | |
| Video-LLaMAParameter count=7B2026.06 | 71.6 | 53.3 | 32.9 | — | |
| HowToCaption2025.03 | 65.3 | — | — | — | |
| COSA-BParameters=17M2023.06 | 64.7 | 48.5 | — | — | |
| COSA-BParameters=5M2023.06 | 64.3 | 48.1 | — | — | |
| Macaw-LLMParameter count=7B2026.06 | 63 | 47.2 | 27.9 | — | |
| HiTeAParameters=5M2023.06 | 62.5 | — | — | — | |
| LAVENDERParameters=30M2023.06 | 60.1 | — | — | — | |
| MV-GPTDecoding=false, E2E=true, 3D Action=ViViT2023.09 | 60 | 48.9 | 38.7 | 64 | |
| Prior SOTA2022.05 | 60 | — | — | — | |
| MV-GPTwith subtitle as additional input=true2022.05 | 60 | 48.9 | — | — | |
| PandaGPTParameter count=13B2026.06 | 59.4 | 42 | 23.9 | — | |
| VIOLETV2Parameters=5M2023.06 | 58 | — | — | — | |
| CoCap (ViT-L/14)Decoding=false, E2E=true, 3D Action=CLIP2023.09 | 57.2 | 44.4 | 30.3 | 63.4 | |
| CoCapData Type=I-frame+MV+Res, Model Time=178 ms, Total Inference Time=178 ms2023.09 | 56.2 | — | — | — | |
| CoCap (ViT-B/16)Decoding=false, E2E=true, 3D Action=CLIP2023.09 | 56.2 | 43.1 | 29.8 | 62.7 | |
| SwinBERT2025.03 | 55.9 | — | — | — | |
| CoCapData Type=I-frame+MV, Model Time=153 ms, Total Inference Time=153 ms2023.09 | 55.3 | — | — | — | |
| AV-LLMParameter count=13B2026.06 | 55.1 | 39 | 21.3 | — | |
| CoCapData Type=I-frame, Model Time=146 ms, Total Inference Time=146 ms2023.09 | 54.1 | — | — | — | |
| SwinBERTData Type=RGB Video Frames, Model Time=339 ms, Total Inference Time=339 ms2023.09 | 53.8 | — | — | — | |
| SwinBERTDecoding=true, E2E=true, 3D Action=VidSwin2023.09 | 53.8 | 41.9 | 29.9 | 62.1 | |
| SwinBERT2024.11 | 53.8 | — | 29.9 | — | |
| SwinBERT2022.05 | 53.8 | 41.9 | — | — | |
| OpenBookDecoding=true, E2E=false, 2D Appearance=IncepResnetV2, 3D Action=C3D2023.09 | 52.9 | 42.8 | 29.3 | 61.7 | |
| OpenBook2022.05 | 52.9 | 33.9 | — | — | |
| OneLLMParameter count=7B2026.06 | 52.8 | 37.9 | 18.7 | — | |
| HMNData Type=RGB Video Frames, Feature Extraction Time=2,710 ms, Model Time=108 ms, Total Inference Time=2,818 ms2023.09 | 51.5 | — | — | — | |
| HMNDecoding=false, E2E=false, 2D Appearance=IncepResnetV2, 3D Action=C3D, Object Detection=FasterRCNN2023.09 | 51.5 | 43.5 | 29 | 62.7 | |
| MGRMPDecoding=false, E2E=false, 2D Appearance=IncepResnetV2, 3D Action=C3D2023.09 | 51.4 | 41.7 | 28.9 | 62.1 | |
| SAATDecoding=false, E2E=false, 2D Appearance=IncepResnetV2, 3D Action=C3D2023.09 | 51 | 39.9 | 27.7 | 61.2 | |
| SAAT2022.05 | 51 | 39.9 | — | — | |
| ORG-TRLDecoding=false, E2E=false, 2D Appearance=IncepResnetV2, 3D Action=C3D, Object Detection=FasterRCNN2023.09 | 50.9 | 43.6 | 28.8 | 62.1 | |
| ORG-TRL2022.05 | 50.9 | 43.6 | — | — | |
| UniVLDecoding=false, E2E=false, 3D Action=S3D2023.09 | 49.9 | 42.2 | 28.8 | 61.2 | |
| UniVL2024.11 | 49.9 | — | 28.2 | — | |
| SGNData Type=RGB Video Frames, Feature Extraction Time=303 ms, Model Time=275 ms, Total Inference Time=578 ms2023.09 | 49.5 | — | — | — | |
| SGNDecoding=false, E2E=false, 2D Appearance=ResNet101, 3D Action=C3D2023.09 | 49.5 | 40.8 | 28.3 | 60.8 | |
| PMI-CAPDecoding=false, E2E=false, 2D Appearance=IncepResnetV2, 3D Action=C3D2023.09 | 49.4 | 42.1 | 28.7 | — | |
| PMI-CAP2022.05 | 49.4 | 42.1 | — | — | |
| POS+VCT2022.05 | 49.1 | 42.3 | — | — | |
| POS+CG2022.05 | 48.7 | 42 | — | — | |
| Support-set2022.05 | 48.6 | 38.9 | — | — | |
| MGSA2022.05 | 47.5 | 42.4 | — | — | |
| SibNet2022.05 | 47.5 | 40.9 | — | — | |
| STG-KDDecoding=false, E2E=false, 2D Appearance=ResNet101, 3D Action=I3D, Object Detection=FasterRCNN2023.09 | 47.1 | 40.5 | 28.3 | 60.9 | |
| STG-KD2022.05 | 47.1 | 40.5 | — | — | |
| OA-BTG2022.05 | 46.9 | 41.4 | — | — |