Page Retrieval on MMDocIR 1.0 (full)
64.7Macro R@1Qwen3-VL-8B
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen3-VL-8BScoring=LVLM attn., Attention strategy=selected heads (MMRetHeads)2026.05 | 64.7 | 83.8 | 88.9 | 64.5 | 83.6 | 88.9 | |
| Qwen3-VL-8BScoring=LVLM attn., Attention strategy=all-head attention2026.05 | 61.8 | 81.2 | 87.5 | 61.2 | 81.2 | 87.7 | |
| Col-Phi3MMDocIRScoring=Image2026.05 | 57 | 76.3 | 82.2 | 57.1 | 76.8 | 83 | |
| DPR-Phi3MMDocIRScoring=Image2026.05 | 54.1 | 73.5 | 81.1 | 53.7 | 74.3 | 81.8 | |
| ColPaliScoring=Image2026.05 | 53 | 74.7 | 80.8 | 52.7 | 75 | 81 | |
| Gemma3-12BScoring=LVLM attn., Attention strategy=selected heads (MMRetHeads)2026.05 | 51.8 | 73.4 | 80.2 | 52.4 | 73.4 | 80.7 | |
| DSEdocmatixScoring=Image2026.05 | 50.2 | 71.4 | 79.5 | 50.1 | 71.8 | 80.1 | |
| Gemma3-12BScoring=LVLM attn., Attention strategy=all-global attention2026.05 | 49.4 | 71 | 78.6 | 50 | 70.8 | 78.9 | |
| DSEwiki-ssScoring=Image2026.05 | 48 | 70.6 | 78.5 | 47.5 | 71.4 | 79.2 | |
| ColBERTScoring=VLM-text2026.05 | 45.8 | 64.9 | 72.3 | 44.9 | 64.8 | 72.3 | |
| ContrieverScoring=VLM-text2026.05 | 40.9 | 60.6 | 69.2 | 39.7 | 59.7 | 68.3 | |
| E5Scoring=VLM-text2026.05 | 40.8 | 60.3 | 69.1 | 39.5 | 59.3 | 67.9 | |
| BGEScoring=VLM-text2026.05 | 40.6 | 59.7 | 68.4 | 39.6 | 59.6 | 68.5 | |
| GTEScoring=VLM-text2026.05 | 38.9 | 58.7 | 67.6 | 37.9 | 58.3 | 67.2 | |
| DPRScoring=VLM-text2026.05 | 27.2 | 46.3 | 57.8 | 26.9 | 46.2 | 57.8 |