Multimodal Video Retrieval on MMEB Video V2 (test)
78.4CLS Hit@1Qwen3-VL-Embedding
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Qwen3-VL-EmbeddingBackbone=Qwen3-VL, Data Access Setting=Closed-Source or w/ Additional Data2026.04 | 78.4 | 71 | 58.7 | 56.1 | 67.1 | |
| Ops-MM-EmbedBackbone=Qwen2-VL, Data Access Setting=Closed-Source or w/ Additional Data2026.04 | 59.7 | 62.2 | 45.7 | 43.2 | 53.8 | |
| Seed-1.6-embeddingBackbone=Seed1.6-flash, Data Access Setting=Closed-Source or w/ Additional Data2026.04 | 55 | 60.9 | 51.3 | 53.5 | 55.3 | |
| RIMEBackbone=Qwen2-VL, Model Size Group=~ 7B Model Size2026.04 | 52.6 | 62 | 38.4 | 41.6 | 49.4 | |
| Think-Then-EmbedBackbone=Qwen2-VL, Data Access Setting=Closed-Source or w/ Additional Data2026.04 | 49.1 | 60.6 | 36.4 | 37.2 | 46.8 | |
| UME-R1Backbone=Qwen2-VL, Model Size Group=~ 7B Model Size2026.04 | 48.6 | 60.7 | 38.2 | 39.3 | 47.5 | |
| RIMEBackbone=Qwen2-VL, Model Size Group=~ 2B Model Size2026.04 | 48 | 52.1 | 33.6 | 39.2 | 43.7 | |
| UME-R1Backbone=Qwen2-VL, Model Size Group=~ 2B Model Size2026.04 | 44.3 | 51.2 | 32.9 | 39.7 | 42.2 | |
| VLM2Vec-V2Backbone=Qwen2-VL, Model Size Group=~ 2B Model Size2026.04 | 39.3 | 34.3 | 28.8 | 38.5 | 34.9 | |
| VLM2VecBackbone=Qwen2-VL, Model Size Group=~ 7B Model Size2026.04 | 39.1 | 30 | 29 | 40.6 | 34 | |
| DUMEBackbone=Qwen2-VL, Model Size Group=~ 2B Model Size2026.04 | 37.7 | 46.6 | 17.1 | 30 | 33.2 | |
| GMEBackbone=Qwen2-VL, Model Size Group=~ 7B Model Size2026.04 | 37.4 | 50.4 | 28.4 | 38.2 | 38.6 | |
| CAFeBackbone=LLaVA-OV, Model Size Group=~ 7B Model Size2026.04 | 35.8 | 58.7 | 34.4 | 39.5 | 42.4 | |
| GMEBackbone=Qwen2-VL, Model Size Group=~ 2B Model Size2026.04 | 34.9 | 42 | 25.6 | 32.4 | 33.9 | |
| VLM2VecBackbone=Qwen2-VL, Model Size Group=~ 2B Model Size2026.04 | 33.4 | 30.5 | 20.6 | 33 | 29 | |
| LamRABackbone=Qwen2.5-VL, Model Size Group=~ 7B Model Size2026.04 | 32.9 | 42.6 | 23.2 | 37.6 | 33.7 | |
| DUMEBackbone=Qwen2-VL, Model Size Group=~ 7B Model Size2026.04 | 32.9 | 47.4 | 8.6 | 28 | 29.4 | |
| ColPali-V1.3Backbone=PaliGemma, Model Size Group=~ 2B Model Size2026.04 | 26.7 | 37.8 | 21.6 | 25.5 | 28.2 |