Nugget-based retrieval on CRUX DUC04
73.8Precision@10Qwen3-Embed
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen3-EmbedSize=8B2026.05 | 73.8 | 75.8 | 60.8 | 66.4 | |
| Qwen3-EmbedSize=0.6B2026.05 | 69.4 | 72 | 55.9 | 61.4 | |
| CoveR (w/o pFT)Backbone=ModernBERT-base, Size=149M, Pre-finetuning=None2026.05 | 69.4 | 72.1 | 56.2 | 61.8 | |
| CoveR (pFT on MS)Backbone=ModernBERT-base, Size=149M, Pre-finetuning=MSMARCO2026.05 | 68.4 | 71 | 57.6 | 62.7 | |
| SPLADE-v3Size=110M2026.05 | 68 | 70.4 | 55.8 | 62.4 | |
| Nomic-EmbedSize=149M2026.05 | 65.4 | 66.8 | 53.2 | 58.8 | |
| MMR (λ = .99)Backbone=ModernBERT-base, Size=149M2026.05 | 63 | 64.8 | 52.1 | 59.1 | |
| Relevance (Rel.)Backbone=ModernBERT-base, Size=149M2026.05 | 62.8 | 64.8 | 51.9 | 58.4 | |
| Unsupervised (Unsup.)Backbone=ModernBERT-base, Size=149M2026.05 | 62.2 | 64.5 | 49.8 | 56.3 | |
| CoveR (pFT on SCOPE-flt)Backbone=ModernBERT-base, Size=149M, Pre-finetuning=SCOPE-flt2026.05 | 62.2 | 65.2 | 53.5 | 58.2 | |
| + MultiQ (RRF)Backbone=ModernBERT-base, Size=149M2026.05 | 58.8 | 61.3 | 48.8 | 57.1 | |
| + MultiQ (SimSum)Backbone=ModernBERT-base, Size=149M2026.05 | 57.6 | 58.4 | 47.1 | 56.3 | |
| + MultiQ (RRB)Backbone=ModernBERT-base, Size=149M2026.05 | 54.2 | 57.4 | 47 | 55.7 | |
| BM25Size=-2026.05 | 51.4 | 53 | 44.5 | 54.4 |