Video Summarization Metric Evaluation on MLVU VS Eval
0.6465Kendall's tau_bQEVA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| QEVADirection=QA-based, Reference Summary=No, Video Used=Yes2026.04 | 0.6465 | 0.6407 | 0.7326 | |
| Video-LMM JudgeDirection=LLM-based, Reference Summary=No, Video Used=Yes2026.04 | 0.5376 | 0.5441 | 0.681 | |
| MLVUDirection=LLM-based, Reference Summary=Yes, Video Used=No2026.04 | 0.5284 | 0.5309 | 0.6738 | |
| METEORDirection=Similarity-based, Reference Summary=Yes, Video Used=No2026.04 | 0.2663 | 0.2689 | 0.3822 | |
| CIDErDirection=Similarity-based, Reference Summary=Yes, Video Used=No2026.04 | 0.2401 | 0.2435 | 0.3612 | |
| SPICEDirection=Similarity-based, Reference Summary=Yes, Video Used=No2026.04 | 0.2287 | 0.2312 | 0.3489 | |
| BLEU-4Direction=Rule-based, Reference Summary=Yes, Video Used=No2026.04 | 0.2113 | 0.1578 | 0.2798 | |
| BLEU-3Direction=Rule-based, Reference Summary=Yes, Video Used=No2026.04 | 0.1994 | 0.1836 | 0.2836 | |
| BERTscoreDirection=Similarity-based, Reference Summary=Yes, Video Used=No2026.04 | 0.1415 | 0.1428 | 0.1987 | |
| ROUGE-LDirection=Rule-based, Reference Summary=Yes, Video Used=No2026.04 | 0.1094 | 0.1104 | 0.1571 | |
| BLEU-2Direction=Rule-based, Reference Summary=Yes, Video Used=No2026.04 | 0.0673 | 0.068 | 0.1045 | |
| Video-Summary SimilarityDirection=Similarity-based, Reference Summary=No, Video Used=Yes2026.04 | 0.0278 | 0.0266 | 0.0401 | |
| BLEU-1Direction=Rule-based, Reference Summary=Yes, Video Used=No2026.04 | 0.0217 | 0.0219 | 0.0431 |