Multimodal Document Information Retrieval on MMDocIR (test)
70.1Macro Recall@1Rank-Nexus
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Rank-NexusProtocol=Reranking from ColQwen2026.01 | 70.1 | 86.96 | 87.53 | 67.1 | 84.97 | 87 | |
| MM-R5Protocol=Reranking from ColQwen2026.01 | 69.51 | 85.2 | 88.42 | 67.59 | 84.01 | 87.55 | |
| Qwen2.5-VL-7B-sftProtocol=Reranking from ColQwen2026.01 | 66.73 | 84.75 | 88.28 | 64.98 | 83.66 | 87.46 | |
| ColQwenProtocol=Retriever-only2026.01 | 64.81 | 83.31 | 87.66 | 63.54 | 82.13 | 86.67 | |
| Qwen2.5-VL-7B-cotProtocol=Reranking from ColQwen, Chain-of-thought=true2026.01 | 64.79 | 81.79 | 86.7 | 63.36 | 81.03 | 86.04 | |
| Qwen2.5-VL-7BProtocol=Reranking from ColQwen2026.01 | 62.3 | 82.43 | 86.75 | 60.12 | 80.69 | 85.55 | |
| Gemma3-12B-cotProtocol=Reranking from ColQwen, Chain-of-thought=true2026.01 | 54.03 | 77.29 | 82.92 | 51.66 | 75.36 | 81.53 | |
| GMEProtocol=Retriever-only2026.01 | 54 | 76.03 | 83.08 | 54.21 | 76.03 | 83.77 | |
| DSEProtocol=Retriever-only2026.01 | 51.09 | 71.94 | 79.25 | 50.23 | 72.34 | 79.71 | |
| Gemma3-12BProtocol=Reranking from ColQwen2026.01 | 47.43 | 66.75 | 72.78 | 44.56 | 63.53 | 70.25 | |
| E5-VProtocol=Retriever-only2026.01 | 42.01 | 64.7 | 73.89 | 42.49 | 65.78 | 75.19 | |
| RagVLProtocol=Reranking from ColQwen2026.01 | 38.14 | 64.62 | 76.67 | 34.11 | 62.06 | 74.97 | |
| CLIPProtocol=Retriever-only2026.01 | 33.34 | 54.28 | 64.52 | 32.27 | 53.37 | 63.5 |