Multimodal Retrieval on MMEB Video V2
51.6CLS AccuracyUME-R1
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| UME-R1Backbone=Qwen2-VL, Parameter Scale=7B, Oracle Evaluation Mode=true2025.11 | 51.6 | 67.2 | 39.6 | 49.6 | 52.2 | |
| UME-R1Backbone=Qwen2-VL, Parameter Scale=7B2025.11 | 48.6 | 60.7 | 38.2 | 39.3 | 47.5 | |
| UME-R1Backbone=Qwen2-VL, Parameter Scale=2B, Oracle Evaluation Mode=true2025.11 | 47 | 58.7 | 37.2 | 48.8 | 47.9 | |
| UME-R1Backbone=Qwen2-VL, Parameter Scale=2B2025.11 | 44.3 | 51.2 | 32.9 | 39.7 | 42.2 | |
| LamRABackbone=Qwen2-VL, Parameter Scale=7B2025.11 | 39.3 | 42.6 | 24.3 | 34.6 | 35.2 | |
| VLM2Vec-V2Backbone=Qwen2-VL, Parameter Scale=2B2025.11 | 39.3 | 34.3 | 28.8 | 38.5 | 34.9 | |
| VLM2VecBackbone=Qwen2-VL, Parameter Scale=7B2025.11 | 39.1 | 30 | 29 | 40.6 | 34 | |
| DUMEBackbone=Qwen2-VL, Parameter Scale=2B2025.11 | 37.7 | 46.6 | 17.1 | 30 | 33.2 | |
| GMEBackbone=Qwen2-VL, Parameter Scale=7B2025.11 | 37.4 | 50.4 | 28.4 | 38.2 | 38.6 | |
| CAFeBackbone=LLaVA-OV, Parameter Scale=7B2025.11 | 35.8 | 58.7 | 34.4 | 39.5 | 42.4 | |
| GMEBackbone=Qwen2-VL, Parameter Scale=2B2025.11 | 34.9 | 42 | 25.6 | 32.4 | 33.9 | |
| VLM2VecBackbone=Qwen2-VL, Parameter Scale=2B2025.11 | 33.4 | 30.5 | 20.6 | 33 | 29 | |
| LamRABackbone=Qwen2.5-VL, Parameter Scale=7B2025.11 | 32.9 | 42.6 | 23.2 | 37.6 | 33.7 | |
| DUMEBackbone=Qwen2-VL, Parameter Scale=7B2025.11 | 32.9 | 47.4 | 8.6 | 28 | 29.4 | |
| ColPali-V1.3Backbone=PaliGemma, Parameter Scale=3B2025.11 | 26.7 | 37.8 | 21.6 | 25.5 | 28.2 |