Nugget Coverage Reranking on CRUX-MDS DUC 2004 (test)
88.2nDCGLANCER_Q*
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LANCER_Q*First-stage retrieval=Qwen3-Embed, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 88.2 | 86.8 | 82.9 | 80.1 | |
| LANCER_Q*First-stage retrieval=LSR, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 86.3 | 84.8 | 81.3 | 79.3 | |
| PointwiseFirst-stage retrieval=Qwen3-Embed, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 83.9 | 83.8 | 63 | 70.2 | |
| PointwiseFirst-stage retrieval=LSR, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 83.1 | 82.4 | 63.2 | 70.6 | |
| ListwiseFirst-stage retrieval=Qwen3-Embed, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 81.7 | 81.4 | 63.2 | 67.5 | |
| LANCERFirst-stage retrieval=Qwen3-Embed, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 80.8 | 80 | 64.4 | 68.5 | |
| ListwiseFirst-stage retrieval=LSR, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 80.6 | 79.8 | 61.1 | 67.1 | |
| LANCER_Q*First-stage retrieval=BM25, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 80.3 | 76.6 | 73.7 | 74.6 | |
| LANCERFirst-stage retrieval=LSR, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 78.9 | 78 | 63.5 | 68.8 | |
| ListwiseFirst-stage retrieval=BM25, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 77.1 | 74.6 | 60.3 | 64.9 | |
| PointwiseFirst-stage retrieval=BM25, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 76.1 | 73.4 | 58.6 | 65.6 | |
| Qwen3-Embedtype=First-stage retrieval2026.01 | 75.9 | 73.8 | 60.8 | 66.8 | |
| SetwiseFirst-stage retrieval=Qwen3-Embed, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 75.3 | 72.4 | 61.9 | 66.6 | |
| LANCERFirst-stage retrieval=BM25, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 73.8 | 72.4 | 60.5 | 66.4 | |
| SetwiseFirst-stage retrieval=LSR, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 71.3 | 68.4 | 58.4 | 64.8 | |
| LSRtype=First-stage retrieval2026.01 | 70.4 | 68 | 55.8 | 64 | |
| SetwiseFirst-stage retrieval=BM25, LLM Backbone=Llama-3.3-70B-Instruct2026.01 | 69.2 | 64 | 57.8 | 63.5 | |
| BM25type=First-stage retrieval2026.01 | 53 | 51.4 | 44.5 | 54.1 |