Page-level reranking on MMDocIR 1.0 (test)
92.6Recall (Resolution)ZipRerank
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ZipRerankFirst-stage retriever=ColQwen, Time (s)=0.36, Recall Level=Recall@52026.05 | 92.6 | — | — | 89.3 | 90 | 89.9 | 95.1 | 91.3 | 81.8 | 94.1 | 93.2 | 97 | 67.9 | 89.3 | 89.1 | |
| GPT-5-miniType=VLM, Time (s)=23.382026.05 | 89.1 | — | 88.3 | — | 95.3 | 85.5 | 94 | 87.3 | 85.7 | 89.1 | 89.4 | 96.2 | 68.6 | 88 | 88.3 | |
| ZipRerankFirst-stage retriever=ColQwen, Time (s)=0.36, Recall Level=Recall@32026.05 | 87.9 | — | 84.9 | — | 86.9 | 86.9 | 90.3 | 86.8 | 77 | 89.3 | 90.5 | 95.5 | 57.7 | 84.9 | 84.4 | |
| ZipRerankType=Reranker, Time (s)=0.362026.05 | 86.7 | — | 84.5 | — | 89.3 | 88 | 90.2 | 87.5 | 79.2 | 87.3 | 87.8 | 95.1 | 56.9 | 84.8 | 84.5 | |
| ColQwenFirst-stage retriever=Self, Recall Level=Recall@52026.05 | 85.9 | — | — | 84.1 | 92.7 | 81.2 | 92.5 | 75.8 | 69.7 | 82.7 | 89.6 | 95.5 | 75.9 | 84.1 | 83.5 | |
| ZipRerankType=Reranker, Pruning percentage=50%, Time (s)=0.302026.05 | 84.3 | — | 83.4 | — | 88.3 | 85.7 | 90.7 | 84.4 | 77.6 | 86.7 | 88.5 | 91.3 | 56.9 | 83.4 | 83.4 | |
| GPT-5-nanoType=VLM, Time (s)=27.612026.05 | 82.9 | — | 79.1 | — | 86.9 | 84.2 | 84.3 | 77.2 | 72.9 | 83.7 | 81.3 | 93.9 | 48.9 | 79.6 | 79.1 | |
| ColQwenFirst-stage retriever=Self, Recall Level=Recall@32026.05 | 79.4 | — | 79 | — | 85.4 | 76.7 | 85.9 | 70.3 | 64.7 | 77.3 | 84.9 | 93.2 | 72.3 | 79 | 78.2 | |
| MM-R5Type=Reranker, Time (s)=3.822026.05 | 78.2 | — | 79 | — | 85 | 80.6 | 86.6 | 71.6 | 69.4 | 80.2 | 77.6 | 89.8 | 72.3 | 79.1 | 79 | |
| LamRA (Listwise)Type=Reranker, Listwise evaluation=true, Time (s)=0.532026.05 | 76.4 | — | 77.8 | — | 73.9 | 77.2 | 85.6 | 77.2 | 68.9 | 82.2 | 76.7 | 86 | 71.5 | 77.6 | 77.8 | |
| Qwen3-VL-8B-InstructType=VLM, Time (s)=2.712026.05 | 75 | — | 70.4 | — | 58.9 | 76 | 77.8 | 69.8 | 65.4 | 76.7 | 74.9 | 86 | 32.1 | 69.3 | 70.4 | |
| UniME (Listwise)Type=Reranker, Listwise evaluation=true, Time (s)=0.242026.05 | 74.1 | — | 71.4 | — | 63.9 | 79.2 | 79 | 66.2 | 62.7 | 77.8 | 73.1 | 81.4 | 51.8 | 70.9 | 71.4 | |
| DSEwiki-ssRole=First-stage retriever2026.05 | 72.7 | — | 70.2 | — | 63.9 | 77.1 | 78.7 | 63.9 | 61.3 | 73.7 | 71.3 | 81.4 | 51.1 | 69.5 | 70.2 | |
| ZipRerankFirst-stage retriever=ColQwen, Time (s)=0.36, Recall Level=Recall@12026.05 | 66.3 | 66.1 | — | — | 67.1 | 65.7 | 70.1 | 70.3 | 56.2 | 71.5 | 72.4 | 79.2 | 42.3 | 66.1 | 65.1 | |
| MM-R5First-stage retriever=ColQwen, Time (s)=3.74, Recall Level=Recall@12026.05 | 65.3 | 68.5 | — | — | 68.3 | 64.9 | 69.1 | 64.7 | 58.1 | 66 | 70.6 | 84.5 | 73.7 | 68.5 | 67.4 | |
| GPT-5-miniFirst-stage retriever=ColQwen, Time (s)=24.70, Recall Level=Recall@12026.05 | 64.8 | 70.9 | — | — | 66.6 | 67.2 | 74.3 | 71.3 | 60.9 | 70.9 | 76 | 85.2 | 71.5 | 70.9 | 70.1 | |
| GPT-5-nanoFirst-stage retriever=ColQwen, Time (s)=27.32, Recall Level=Recall@12026.05 | 62.4 | 63.6 | — | — | 60.6 | 64.1 | 66.3 | 60.7 | 51.1 | 67.1 | 70.6 | 79.9 | 53.3 | 63.6 | 62.5 | |
| ZipRerank−50%First-stage retriever=ColQwen, Time (s)=0.31, Recall Level=Recall@12026.05 | 62.4 | 63.9 | — | — | 64.4 | 65.2 | 69.1 | 68.3 | 54 | 65.8 | 70.6 | 75.4 | 43.8 | 63.9 | 63 | |
| LamRA (Listwise)First-stage retriever=ColQwen, Time (s)=0.55, Recall Level=Recall@12026.05 | 60.7 | 65.4 | — | — | 56.8 | 61.7 | 71 | 61.1 | 57.1 | 69 | 70.6 | 76.1 | 70.1 | 65.4 | 65.6 | |
| UniME (Listwise)First-stage retriever=ColQwen, Time (s)=0.25, Recall Level=Recall@12026.05 | 58 | 59.3 | — | — | 56.7 | 63.8 | 56.4 | 51.2 | 47.8 | 60.9 | 61.6 | 75.4 | 61.3 | 59.3 | 57.6 | |
| ColQwenFirst-stage retriever=Self, Recall Level=Recall@12026.05 | 56.2 | 60.4 | — | — | 55.3 | 60.8 | 65.4 | 53.8 | 48.1 | 60.4 | 66.1 | 72.3 | 65.7 | 60.4 | 59.9 | |
| Qwen3-VL-8B-InstructFirst-stage retriever=ColQwen, Time (s)=2.47, Recall Level=Recall@12026.05 | 35.2 | 29.8 | — | — | 16.4 | 41.6 | 35.9 | 28.7 | 18.9 | 37.2 | 43.5 | 38.6 | 1.5 | 29.8 | 29.6 | |
| Llama-3.2-11B-VisionType=VLM, Time (s)=5.512026.05 | 7.9 | — | 20.5 | — | 25.9 | 9.1 | 53.1 | 6.6 | 10.3 | 8.4 | 9.5 | 21.6 | 3.6 | 15.6 | 20.5 | |
| Llama-3.2-11B-VisionFirst-stage retriever=ColQwen, Time (s)=5.51, Recall Level=Recall@12026.05 | 0 | 0.9 | — | — | 0.9 | 0 | 3 | 0 | 0.7 | 1.3 | 0.9 | 0.8 | 1.5 | 0.9 | 1.2 |