Multimodal Reasoning-Intensive Retrieval on MR2-Bench
49.7nDCG@10 (Bio.)MM-Embed
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MM-EmbedBackbone=NV-Embed, Size=8B2026.04 | 49.7 | 52.2 | 23.7 | 30.4 | 17.4 | 47.5 | 43 | 21.6 | 48.4 | 22.8 | 0.2 | 5.9 | 30.2 | |
| Seed-1.6-embedBackbone=Seed1.6-flash, Size=–2026.04 | 40.6 | 38.1 | 31.8 | 27.9 | 17.8 | 37.2 | 56.1 | 26.1 | 65.2 | 17.3 | 0.9 | 9.2 | 30.7 | |
| VLM2VecBackbone=Qwen2-VL, Size=2B2026.04 | 39.4 | 39.4 | 19.9 | 20.3 | 9 | 35.7 | 51.4 | 14.2 | 35.1 | 13.9 | 0.6 | 5.9 | 23.7 | |
| RIMEBackbone=Qwen2-VL, Size=7B2026.04 | 38.5 | 32 | 25.6 | 20.4 | 7.1 | 23.3 | 64.2 | 24.2 | 50 | 40.4 | 1.2 | 15.1 | 28.6 | |
| UME-R1Backbone=Qwen2-VL, Size=7B2026.04 | 38.2 | 32.6 | 25.4 | 19.6 | 8.7 | 26.3 | 55.7 | 23.2 | 49.2 | 37.7 | 0.4 | 12.2 | 27.5 | |
| GMEBackbone=Qwen2-VL, Size=2B2026.04 | 34.3 | 39.5 | 19 | 19.3 | 7.7 | 28.6 | 37 | 7.2 | 39.4 | 15.7 | 0.2 | 11.1 | 21.6 | |
| CLIPBackbone=ViT-L/14, Size=0.4B2026.04 | 32.9 | 30.6 | 14.1 | 14.9 | 3.5 | 33.2 | 13 | 5.6 | 49.3 | 20.9 | 0.2 | 5.1 | 18.6 | |
| BGE-VLBackbone=BGE-VL, Size=2B2026.04 | 29.4 | 18.4 | 10.5 | 19.5 | 7.1 | 19.7 | 50.8 | 14.3 | 48 | 6.5 | 0 | 0.8 | 19.5 |