Video Captioning on VATEX
99.5CIDErVAST
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| VASTmodality track=Multi-modal, SCST finetuning=true2023.05 | 99.5 | — | — | — | — | — | — | — | — | — | |
| VAST(1.3B)Sample=442M, Audio or subtitle modalities=true, SCST finetuning=true2023.05 | 99.5 | — | — | — | — | 45 | — | — | — | — | |
| Method [7]modality track=Multi-modal, SCST finetuning=true2023.05 | 95.8 | — | — | — | — | — | — | — | — | — | |
| VALOR-LSample=433.5M, Audio or subtitle modalities=true, SCST finetuning=true2023.05 | 95.8 | — | — | — | — | 45.6 | — | — | — | — | |
| VALORL#Example=33.5M, Mod=V+A, Reinforcement Learning=true2023.04 | 95.8 | — | — | — | — | 45.6 | 29.4 | 57.4 | — | — | |
| VALORzero-shot=false2023.12 | 95.1 | — | — | — | — | — | — | — | — | — | |
| Method [19]modality track=Vision-only, SCST finetuning=true2023.05 | 94.5 | — | — | — | — | — | — | — | — | — | |
| GIT2(5.1B)Sample=12.9B, Audio or subtitle modalities=false, SCST finetuning=true2023.05 | 94.5 | — | — | — | — | 42.7 | — | — | — | — | |
| GIT2 (5.1B)#Example=12.9B, Mod=V, Reinforcement Learning=true2023.04 | 94.5 | — | — | — | — | 42.7 | 28.8 | 56.5 | — | — | |
| GIT2optimization=Self-Critical Sequence Training2023.05 | 94 | — | — | — | — | — | — | — | — | — | |
| GIT2number of frames=162023.10 | 94 | — | — | — | — | — | — | — | — | — | |
| GITSample=1.7B, Audio or subtitle modalities=false, SCST finetuning=true2023.05 | 91.5 | — | — | — | — | 41.6 | — | — | — | — | |
| GIT#Example=800M, Mod=V, Reinforcement Learning=true2023.04 | 91.5 | — | — | — | — | 41.6 | 28.1 | 55.4 | — | — | |
| CLIP4Caption++Sample=400M, Audio or subtitle modalities=true, SCST finetuning=true2023.05 | 85.7 | — | — | — | — | 40.6 | — | — | — | — | |
| VideoCoCaEvaluation Protocol=finetuning2022.12 | 77.8 | — | — | — | — | 39.7 | — | 54.5 | — | — | |
| CoDi2023.12 | 74.4 | — | — | — | — | — | — | — | — | — | |
| VALORB#Example=6.5M, Mod=V+A2023.04 | 73.9 | — | — | — | — | 41.9 | 26.6 | 54.6 | — | — | |
| SwinBERTSample=None, Audio or subtitle modalities=false, SCST finetuning=false2023.05 | 73 | — | — | — | — | 38.7 | — | — | — | — | |
| SwinBERTMod=V2023.04 | 73 | — | — | — | — | 38.7 | 26.2 | 53.2 | — | — | |
| GITL#Example=20M, Mod=V, Reinforcement Learning=true2023.04 | 72.5 | — | — | — | — | 41.6 | 26.2 | 54.3 | — | — | |
| VALOR#Example=5.5M, Mod=V2023.04 | 71.6 | — | — | — | — | 40.7 | 26.1 | 53.8 | — | — | |
| SMPFFSample=None, Audio or subtitle modalities=true, SCST finetuning=false2023.05 | 70.5 | — | — | — | — | 39.7 | — | — | — | — | |
| SMPFFMod=V+A2023.04 | 70.5 | — | — | — | — | 39.7 | 26 | 53.6 | — | — | |
| PaLI-XNumber of frames=162023.05 | 69.3 | — | — | — | — | — | — | — | — | — | |
| PaLI-3number of frames=162023.10 | 66.9 | — | — | — | — | — | — | — | — | — | |
| PaLI-XNumber of frames=82023.05 | 66 | — | — | — | — | — | — | — | — | — | |
| VALUESample=136M, Audio or subtitle modalities=true, SCST finetuning=false2023.05 | 58.1 | — | — | — | — | — | — | — | — | — | |
| VALUE#Example=136M, Mod=V+S2023.04 | 58.1 | — | — | — | — | 32.9 | 24 | 50 | — | — | |
| MGRMPVenue=ICCV’21, Method Type=Autoregressive2026.04 | 57.6 | — | — | — | — | 34.2 | 23.5 | — | — | 50.3 | |
| OpenBookMod=V2023.04 | 57.5 | — | — | — | — | 33.9 | 23.7 | 50.2 | — | — | |
| Tarsier-34BZero-shot=true2024.06 | 57.1 | — | — | — | — | — | — | — | — | — | |
| KG-VCNVenue=PR’25, Method Type=Autoregressive2026.04 | 53.3 | — | — | — | — | 33.3 | 22.9 | — | — | 49.5 | |
| Support-set#Example=136M, Mod=V2023.04 | 51.2 | — | — | — | — | 32.8 | 24.4 | 49.1 | — | — | |
| Tarsier-7BZero-shot=true2024.06 | 51.1 | — | — | — | — | — | — | — | — | — | |
| DiffVCVenue=-, Method Type=Non-autoregressive2026.04 | 50.3 | — | — | — | — | 29.5 | 21 | — | — | 48.7 | |
| ORG-TRLMod=V2023.04 | 49.7 | — | — | — | — | 32.1 | 22.2 | 48.9 | — | — | |
| ORG-TRLEvaluation Protocol=finetuning2022.12 | 49.7 | — | — | — | — | 32.1 | — | 48.9 | — | — | |
| ORG-TRLVenue=CVPR’20, Method Type=Autoregressive2026.04 | 49.7 | — | — | — | — | 32.1 | 22.2 | — | — | 48.9 | |
| InternVideo2Zero-shot=true2024.06 | 49.2 | — | — | — | — | — | — | — | — | — | |
| ChatBridge-13Bzero-shot=true2023.12 | 48.9 | — | — | — | — | — | — | — | — | — | |
| R-ConvEDVenue=TOMM’23, Method Type=Autoregressive2026.04 | 48.7 | — | — | — | — | 32.1 | 21.8 | — | — | — | |
| UIO-2XXLModel Size=XXL2023.12 | 45.6 | — | — | — | — | — | — | — | — | — | |
| OneLLM-7Bzero-shot=true2023.12 | 43.8 | — | — | — | — | — | — | — | — | — | |
| SynPOModel Backbone=InternVL2-8B, Fine-tuning Dataset=Sharegpt4video2025.06 | 42.8 | — | — | — | — | — | 20.1 | — | — | — | |
| SynPOModel Backbone=AuroraCap, Fine-tuning Dataset=Pandas-70M2025.06 | 42.6 | — | — | — | — | — | 19.7 | — | — | — | |
| SynPOModel Backbone=AuroraCap, Fine-tuning Dataset=Sharegpt4video2025.06 | 42.5 | — | — | — | — | — | 19.6 | — | — | — | |
| SynPOModel Backbone=AuroraCap, Fine-tuning Dataset=Charades2025.06 | 42.2 | — | — | — | — | — | 19.6 | — | — | — | |
| UIO-2XLModel Size=XL2023.12 | 41.6 | — | — | — | — | — | — | — | — | — | |
| ALSO-NetVenue=TOMM’24, Method Type=Non-autoregressive2026.04 | 40.4 | — | — | — | — | 27.8 | 20.6 | — | — | 46.8 | |
| FlamingoZero-shot=true2024.06 | 39.5 | — | — | — | — | — | — | — | — | — | |
| BaseModel Backbone=AuroraCap, Fine-tuning Dataset=Sharegpt4video2025.06 | 38.4 | — | — | — | — | — | 18.6 | — | — | — | |
| SynPOModel Backbone=LLaVA1.6-7B-video, Fine-tuning Dataset=Sharegpt4video2025.06 | 37.3 | — | — | — | — | — | 18.1 | — | — | — | |
| UIO-2LModel Size=L2023.12 | 37.1 | — | — | — | — | — | — | — | — | — | |
| VideoPrismZero-shot=true2024.06 | 31.7 | — | — | — | — | — | — | — | — | — | |
| NITS-VCEvaluation Protocol=finetuning2022.12 | 24 | — | — | — | — | 20 | — | 42 | — | — | |
| VideoCocaZero-shot=true2024.06 | 22.8 | — | — | — | — | — | — | — | — | — | |
| VideoCoCaEvaluation Protocol=zero-shot2022.12 | 22.8 | — | — | — | — | 9.5 | — | 29.5 | — | — | |
| CoCaEvaluation Protocol=zero-shot2022.12 | 12.4 | — | — | — | — | 6.2 | — | 21 | — | — | |
| FULL CACHEModel Architecture=Qwen2.5-VL-3B2026.04 | 0.5661 | — | — | — | — | 29.79 | 23.3 | 48.96 | 39.67 | — | |
| HYBRIDKVModel Architecture=Qwen2.5-VL-3B2026.04 | 0.5427 | — | — | — | — | 29.36 | 22.86 | 47.72 | 38.55 | — | |
| LOOK-MModel Architecture=Qwen2.5-VL-3B2026.04 | 0.5176 | — | — | — | — | 28.51 | 22.45 | 47.56 | 37.57 | — | |
| SNAPKVModel Architecture=Qwen2.5-VL-3B2026.04 | 0.5072 | — | — | — | — | 27.42 | 21.86 | 47.48 | 36.87 | — | |
| HYBRIDKVModel Architecture=Qwen2.5-VL-7B2026.04 | 0.4774 | — | — | — | — | 22.66 | 22.36 | 43.18 | 33.99 | — | |
| MADAKVModel Architecture=Qwen2.5-VL-3B2026.04 | 0.4708 | — | — | — | — | 26.91 | 20.95 | 45.97 | 35.23 | — | |
| FULL CACHEModel Architecture=Qwen2.5-VL-7B2026.04 | 0.4628 | — | — | — | — | 21.81 | 22.09 | 42.66 | 33.21 | — | |
| LOOK-MModel Architecture=Qwen2.5-VL-7B2026.04 | 0.4571 | — | — | — | — | 21.8 | 21.74 | 42.8 | 33.01 | — | |
| SNAPKVModel Architecture=Qwen2.5-VL-7B2026.04 | 0.4498 | — | — | — | — | 21.46 | 21.25 | 42.4 | 32.52 | — | |
| SPARSEMMModel Architecture=Qwen2.5-VL-3B2026.04 | 0.4352 | — | — | — | — | 27.08 | 20.8 | 45.44 | 34.21 | — | |
| MADAKVModel Architecture=Qwen2.5-VL-7B2026.04 | 0.4306 | — | — | — | — | 20.51 | 21.57 | 42.28 | 31.86 | — | |
| SPARSEMMModel Architecture=Qwen2.5-VL-7B2026.04 | 0.4198 | — | — | — | — | 21.33 | 21.16 | 42.27 | 31.69 | — | |
| FULL CACHEModel Architecture=LLaVA-OV-7B2026.04 | 0.2355 | — | — | — | — | 13.3 | 19 | 37.39 | 23.31 | — | |
| HYBRIDKVModel Architecture=LLaVA-OV-7B2026.04 | 0.2109 | — | — | — | — | 13.74 | 18 | 37 | 22.46 | — | |
| LOOK-MModel Architecture=LLaVA-OV-7B2026.04 | 0.1957 | — | — | — | — | 12.05 | 17.72 | 36.25 | 21.4 | — | |
| SNAPKVModel Architecture=LLaVA-OV-7B2026.04 | 0.1799 | — | — | — | — | 11.61 | 17.04 | 35.84 | 20.62 | — | |
| SPARSEMMModel Architecture=LLaVA-OV-7B2026.04 | 0.177 | — | — | — | — | 11.93 | 16.88 | 35.93 | 20.61 | — | |
| MADAKVModel Architecture=LLaVA-OV-7B2026.04 | 0.1744 | — | — | — | — | 11.15 | 17.48 | 36.01 | 20.52 | — | |
| VASTABackbone=Kinetics2022.08 | — | 86.18 | 97.29 | 85.8 | 7.04 | — | — | — | — | — |