# Layout Retrieval on MMDocIR 1.0 (full)

Source: [Layout Retrieval on MMDocIR 1.0 (full)](<https://www.sota2.com/research/sota/layout-retrieval-on-mmdocir-1-0-full>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-27T10:14:01.491Z

Evaluation of Layout Retrieval performance on the full split of the MMDocIR 1.0 dataset, reporting Macro and Micro Recall at ranks 1, 5, and 10.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Layout Retrieval |
| Dataset (including any stated split) | MMDocIR 1.0 (full) |
| Metric | Macro Recall@1 |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 13 of 13 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Macro Recall@1](<https://www.sota2.com/research/sota/layout-retrieval-on-mmdocir-1-0-full.md?metric=Macro%20Recall%401>): Higher is better; unit: PERCENT.

- [Macro Recall@5](<https://www.sota2.com/research/sota/layout-retrieval-on-mmdocir-1-0-full.md?metric=Macro%20Recall%405>): Higher is better; unit: PERCENT.

- [Macro Recall@10](<https://www.sota2.com/research/sota/layout-retrieval-on-mmdocir-1-0-full.md?metric=Macro%20Recall%4010>): Higher is better; unit: PERCENT.

- [Micro Recall@1](<https://www.sota2.com/research/sota/layout-retrieval-on-mmdocir-1-0-full.md?metric=Micro%20Recall%401>): Higher is better; unit: PERCENT.

- [Micro Recall@5](<https://www.sota2.com/research/sota/layout-retrieval-on-mmdocir-1-0-full.md?metric=Micro%20Recall%405>): Higher is better; unit: PERCENT.

- [Micro Recall@10](<https://www.sota2.com/research/sota/layout-retrieval-on-mmdocir-1-0-full.md?metric=Micro%20Recall%4010>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Macro Recall@1 | Result date | Report paper |
| --- | --- | --- | --- |
| Qwen3-VL-8B | 39 | Not recorded | 6a167ad5dbd7150506f802bb |
| ColPali | 32.7 | Not recorded | 6a167ad5dbd7150506f802bb |
| ColBERT | 31.3 | Not recorded | 6a167ad5dbd7150506f802bb |
| Col-Phi3MMDocIR | 31.1 | Not recorded | 6a167ad5dbd7150506f802bb |
| DPR-Phi3MMDocIR | 29.5 | Not recorded | 6a167ad5dbd7150506f802bb |
| BGE | 28.3 | Not recorded | 6a167ad5dbd7150506f802bb |
| Contriever | 28.3 | Not recorded | 6a167ad5dbd7150506f802bb |
| DSEwiki-ss | 28.2 | Not recorded | 6a167ad5dbd7150506f802bb |
| Gemma3-12B | 28.2 | Not recorded | 6a167ad5dbd7150506f802bb |
| DSEdocmatix | 27.9 | Not recorded | 6a167ad5dbd7150506f802bb |
| E5 | 26.7 | Not recorded | 6a167ad5dbd7150506f802bb |
| GTE | 26.1 | Not recorded | 6a167ad5dbd7150506f802bb |
| DPR | 19.3 | Not recorded | 6a167ad5dbd7150506f802bb |

### Qwen3-VL-8B

Macro Recall@1: 39. Result ID: 6a16c3d6fbc373592c10820f.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

Recorded evaluation configuration: {"Scoring":"LVLM attn.","Attention strategy":"selected heads (MMRetHeads)"}. This record does not assert independent reproduction.

### ColPali

Macro Recall@1: 32.7. Result ID: 6a16c3d6fbc373592c10820c.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77261e209131cea7161f.md>)

Recorded evaluation configuration: {"Scoring":"Image"}. This record does not assert independent reproduction.

### ColBERT

Macro Recall@1: 31.3. Result ID: 6a16c3d6fbc373592c108205.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67a71e209131cea1780b.md>)

Recorded evaluation configuration: {"Scoring":"VLM-text"}. This record does not assert independent reproduction.

### Col-Phi3MMDocIR

Macro Recall@1: 31.1. Result ID: 6a16c3d6fbc373592c10820e.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78a81e209131cea79a85.md>)

Recorded evaluation configuration: {"Scoring":"Image"}. This record does not assert independent reproduction.

### DPR-Phi3MMDocIR

Macro Recall@1: 29.5. Result ID: 6a16c3d6fbc373592c10820d.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78a81e209131cea79a85.md>)

Recorded evaluation configuration: {"Scoring":"Image"}. This record does not assert independent reproduction.

### BGE

Macro Recall@1: 28.3. Result ID: 6a16c3d6fbc373592c108206.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

Recorded evaluation configuration: {"Scoring":"VLM-text"}. This record does not assert independent reproduction.

### Contriever

Macro Recall@1: 28.3. Result ID: 6a16c3d6fbc373592c108208.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6d4b1e209131cea38d50.md>)

[Implementation](<https://github.com/facebookresearch/contriever>)

Recorded evaluation configuration: {"Scoring":"VLM-text"}. This record does not assert independent reproduction.

### DSEwiki-ss

Macro Recall@1: 28.2. Result ID: 6a16c3d6fbc373592c10820a.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76011e209131cea6addd.md>)

Recorded evaluation configuration: {"Scoring":"Image"}. This record does not assert independent reproduction.

### Gemma3-12B

Macro Recall@1: 28.2. Result ID: 6a16c3d6fbc373592c108210.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

Recorded evaluation configuration: {"Scoring":"LVLM attn.","Attention strategy":"selected heads (MMRetHeads)"}. This record does not assert independent reproduction.

### DSEdocmatix

Macro Recall@1: 27.9. Result ID: 6a16c3d6fbc373592c10820b.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76011e209131cea6addd.md>)

Recorded evaluation configuration: {"Scoring":"Image"}. This record does not assert independent reproduction.

### E5

Macro Recall@1: 26.7. Result ID: 6a16c3d6fbc373592c108207.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

Recorded evaluation configuration: {"Scoring":"VLM-text"}. This record does not assert independent reproduction.

### GTE

Macro Recall@1: 26.1. Result ID: 6a16c3d6fbc373592c108209.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

Recorded evaluation configuration: {"Scoring":"VLM-text"}. This record does not assert independent reproduction.

### DPR

Macro Recall@1: 19.3. Result ID: 6a16c3d6fbc373592c108204.

[Reporting paper](<https://www.sota2.com/research/paper/6a167ad5dbd7150506f802bb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa686d1e209131cea1c190.md>)

[Implementation](<https://github.com/facebookresearch/DPR>)

Recorded evaluation configuration: {"Scoring":"VLM-text"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
