Layout Retrieval on MMDocIR 1.0 (full)
39Macro Recall@1Qwen3-VL-8B
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen3-VL-8BScoring=LVLM attn., Attention strategy=selected heads (MMRetHeads)2026.05 | 39 | 64.9 | 74.2 | 39.3 | 67.1 | 76.6 | |
| ColPaliScoring=Image2026.05 | 32.7 | 54 | 62 | 32.5 | 54.3 | 63.2 | |
| ColBERTScoring=VLM-text2026.05 | 31.3 | 54.4 | 61.9 | 31.4 | 56 | 63.7 | |
| Col-Phi3MMDocIRScoring=Image2026.05 | 31.1 | 52.3 | 61.1 | 31.6 | 54.5 | 63.3 | |
| DPR-Phi3MMDocIRScoring=Image2026.05 | 29.5 | 51.6 | 60.2 | 30.2 | 53.9 | 62.8 | |
| BGEScoring=VLM-text2026.05 | 28.3 | 51.8 | 60.3 | 29 | 53.2 | 62.4 | |
| ContrieverScoring=VLM-text2026.05 | 28.3 | 51.7 | 60.4 | 28.9 | 53 | 62.2 | |
| DSEwiki-ssScoring=Image2026.05 | 28.2 | 50.2 | 58.5 | 29.2 | 52.1 | 61.1 | |
| Gemma3-12BScoring=LVLM attn., Attention strategy=selected heads (MMRetHeads)2026.05 | 28.2 | 55.2 | 64.7 | 27.6 | 55.2 | 66.1 | |
| DSEdocmatixScoring=Image2026.05 | 27.9 | 49.8 | 57.5 | 29.1 | 51.9 | 59.9 | |
| E5Scoring=VLM-text2026.05 | 26.7 | 51.1 | 60.4 | 26.4 | 51.8 | 61.2 | |
| GTEScoring=VLM-text2026.05 | 26.1 | 51.3 | 60.7 | 27.1 | 52.3 | 62.2 | |
| DPRScoring=VLM-text2026.05 | 19.3 | 39.8 | 49.5 | 19.2 | 40.4 | 50.5 |