Multimodal Page-level Reranking on MMDocIR v1 (test)
69.2Recall@1GPT-5-mini
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-5-miniType=VLM, Time (s)=23.382026.05 | 69.2 | 65.1 | 74 | 66.2 | 75.5 | 69 | 59.4 | 67.9 | 76 | 82.2 | 65 | 70 | 69.2 | |
| MM-R5Type=Reranker, Time (s)=3.822026.05 | 65.1 | 64.1 | 70.7 | 64.3 | 68 | 58.8 | 56.1 | 66 | 67.9 | 78.4 | 67.2 | 66.1 | 65.1 | |
| LamRA (Listwise)Type=Reranker, Listwise evaluation=true, Time (s)=0.532026.05 | 63.6 | 61.3 | 52.8 | 59 | 67.2 | 63.4 | 56.5 | 65.8 | 66.3 | 72.3 | 69.3 | 63.4 | 63.6 | |
| ZipRerankType=Reranker, Time (s)=0.362026.05 | 63.3 | 61.6 | 65.3 | 64.1 | 67.6 | 65.7 | 56.1 | 70.6 | 68.8 | 76.1 | 46 | 64.2 | 63.3 | |
| ZipRerankType=Reranker, Pruning percentage=50%, Time (s)=0.302026.05 | 62.4 | 64 | 69.9 | 63.3 | 68.5 | 65.7 | 54.3 | 63.6 | 67.9 | 72.3 | 43.1 | 63.3 | 62.4 | |
| GPT-5-nanoType=VLM, Time (s)=27.612026.05 | 59 | 60.6 | 57.6 | 65.4 | 63.9 | 60.6 | 49.7 | 61.3 | 67 | 73.1 | 38.7 | 59.8 | 59 | |
| UniME (Listwise)Type=Reranker, Listwise evaluation=true, Time (s)=0.242026.05 | 49.1 | 55.3 | 46.9 | 64.3 | 43.8 | 46.8 | 43 | 61.1 | 55.5 | 60.6 | 35 | 51.2 | 49.1 | |
| DSEwiki-ssRole=First-stage retriever2026.05 | 45.5 | 46.5 | 42.4 | 52.1 | 47.8 | 43.9 | 39.3 | 50.3 | 48.9 | 49.6 | 39.4 | 46 | 45.5 | |
| Qwen3-VL-8B-InstructType=VLM, Time (s)=2.712026.05 | 26.3 | 30.1 | 8.5 | 31.5 | 36.6 | 21.7 | 16.6 | 31.4 | 36 | 35.2 | 0.7 | 24.8 | 26.3 | |
| Llama-3.2-11B-VisionType=VLM, Time (s)=5.512026.05 | 1.5 | 0.6 | 0.9 | 0 | 4.8 | 0 | 0.2 | 2 | 0 | 1.5 | 0 | 1 | 1.5 |