Information Retrieval on Robust04
46.67P@20CEDR-KNRM
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| CEDR-KNRMInput Representation=BERT (fine-tuned)2019.04 | 46.67 | — | 53.81 | — | — | — | — | |
| CEDR-DRMMInput Representation=BERT (fine-tuned)2019.04 | 45.87 | — | 52.59 | — | — | — | — | |
| CEDR-PACRRInput Representation=BERT (fine-tuned)2019.04 | 45.59 | — | 51.5 | — | — | — | — | |
| PACRRInput Representation=BERT (fine-tuned)2019.04 | 44.92 | — | 51.35 | — | — | — | — | |
| TREC-BestInput Representation=n/a2019.04 | 43.86 | — | 50.3 | — | — | — | — | |
| KNRMInput Representation=BERT (fine-tuned)2019.04 | 42.21 | — | 48.58 | — | — | — | — | |
| NPRFds-DRMMQuery field=Title2018.10 | 40.64 | 29.04 | 45.02 | — | — | — | — | |
| Vanilla BERTInput Representation=BERT (fine-tuned)2019.04 | 40.42 | — | 45.41 | — | — | — | — | |
| NPRFds-DRMMQuery field=Description2018.10 | 40.26 | 28.01 | 45.59 | — | — | — | — | |
| DRMM (QE)Query field=Title2018.10 | 40.02 | 28.76 | 45.49 | — | — | — | — | |
| NPRFds-KNRMQuery field=Description2018.10 | 39.72 | 28 | 44.77 | — | — | — | — | |
| NPRFds-KNRMQuery field=Title2018.10 | 39.26 | 28.46 | 43.27 | — | — | — | — | |
| QL+RM3Query field=Title2018.10 | 38.78 | 28.42 | 43.98 | — | — | — | — | |
| BM25+QEQuery field=Title2018.10 | 38.39 | 29.66 | 43.53 | — | — | — | — | |
| DRMM (QE)Query field=Description2018.10 | 38.22 | 27.11 | 43.92 | — | — | — | — | |
| DRMM_LCHxIDFModel Type=Interaction-Focused Matching, Histogram=Log-Count, Weighting=IDF2017.11 | 38.2 | 27.9 | 43.1 | — | — | — | — | |
| BM25+QEQuery field=Description2018.10 | 38.17 | 29.26 | 43.4 | — | — | — | — | |
| KNRMInput Representation=BERT2019.04 | 38.17 | — | 43.18 | — | — | — | — | |
| DRMM_LCHxTVModel Type=Interaction-Focused Matching, Histogram=Log-Count, Weighting=Term Vector2017.11 | 38.1 | 26.8 | 42.3 | — | — | — | — | |
| RankProb modelInput Representation=Embed2017.04 | 38.02 | 28.37 | 43.89 | — | — | — | — | |
| Rank modelInput Representation=Embed2017.04 | 37.73 | 28.11 | 43.02 | — | — | — | — | |
| SDMInput Representation=n/a2019.04 | 37.49 | — | 43.53 | — | — | — | — | |
| DRMM_LCHxIDFModel Type=Interaction-Focused Matching, Histogram=Log-Count, Weighting=IDF2017.11 | 37.1 | 27.5 | 43.7 | — | — | — | — | |
| BM25Model Type=Traditional Retrieval Baselines2017.11 | 37 | 25.5 | 41.8 | — | — | — | — | |
| QLModel Type=Traditional Retrieval Baselines2017.11 | 36.9 | 25.3 | 41.5 | — | — | — | — | |
| PACRRInput Representation=BERT2019.04 | 36.5 | — | 42 | — | — | — | — | |
| K-NRM (QE)Query field=Title2018.10 | 36.44 | 25.21 | 40.62 | — | — | — | — | |
| DRMMInput Representation=BERT (fine-tuned)2019.04 | 36.41 | — | 41.35 | — | — | — | — | |
| DRMM_CHxIDFModel Type=Interaction-Focused Matching, Histogram=Count, Weighting=IDF2017.11 | 36.2 | 25.9 | 41.2 | — | — | — | — | |
| DRMM_LCHxTVModel Type=Interaction-Focused Matching, Histogram=Log-Count, Weighting=Term Vector2017.11 | 36 | 26.5 | 42.3 | — | — | — | — | |
| DRMM_CHxTVModel Type=Interaction-Focused Matching, Histogram=Count, Weighting=Term Vector2017.11 | 35.7 | 25.3 | 40.7 | — | — | — | — | |
| BM252017.04 | 35.69 | 25.03 | 41.02 | — | — | — | — | |
| PACRRInput Representation=ELMo2019.04 | 35.54 | — | 41.01 | — | — | — | — | |
| PACRRInput Representation=GloVe2019.04 | 35.35 | — | 40.43 | — | — | — | — | |
| KNRMInput Representation=ELMo2019.04 | 35.17 | — | 40.89 | — | — | — | — | |
| QL+RM3Query field=Description2018.10 | 35.06 | 26.86 | 41.5 | — | — | — | — | |
| Score modelInput Representation=Embed2017.04 | 35.01 | 24.23 | 39.99 | — | — | — | — | |
| DRMM_CHxIDFModel Type=Interaction-Focused Matching, Histogram=Count, Weighting=IDF2017.11 | 34.4 | 25.5 | 41 | — | — | — | — | |
| DRMM_CHxTVModel Type=Interaction-Focused Matching, Histogram=Count, Weighting=Term Vector2017.11 | 34.1 | 24.7 | 40.4 | — | — | — | — | |
| KNRMInput Representation=GloVe2019.04 | 34.08 | — | 38.71 | — | — | — | — | |
| BM25Model Type=Traditional Retrieval Baselines2017.11 | 33.7 | 24.1 | 39.9 | — | — | — | — | |
| ConvKNRMInput Representation=GloVe2019.04 | 33.49 | — | 38.06 | — | — | — | — | |
| QLModel Type=Traditional Retrieval Baselines2017.11 | 33.4 | 24.6 | 39.1 | — | — | — | — | |
| K-NRM (QE)Query field=Description2018.10 | 33.04 | 23.8 | 37.85 | — | — | — | — | |
| RankProb modelInput Representation=Sparse2017.04 | 32.5 | 22.46 | 37.63 | — | — | — | — | |
| Rank modelInput Representation=Sparse2017.04 | 32.16 | 22.13 | 36.28 | — | — | — | — | |
| Score modelInput Representation=Sparse2017.04 | 31.8 | 21.41 | 36.04 | — | — | — | — | |
| BM25Input Representation=n/a2019.04 | 31.23 | — | 41.4 | — | — | — | — | |
| RankProb modelInput Representation=Dense2017.04 | 29.66 | 21.92 | 32.78 | — | — | — | — | |
| MP_COSModel Type=Interaction-Focused Matching Baselines2017.11 | 29 | 18.9 | 33 | — | — | — | — | |
| DRMMInput Representation=GloVe2019.04 | 28.92 | — | 30.4 | — | — | — | — | |
| DRMMInput Representation=BERT2019.04 | 28.78 | — | 31.94 | — | — | — | — | |
| DRMMInput Representation=ELMo2019.04 | 28.67 | — | 31.37 | — | — | — | — | |
| Rank modelInput Representation=Dense2017.04 | 28.3 | 19.4 | 33.17 | — | — | — | — | |
| DRMM_NHxIDFModel Type=Interaction-Focused Matching, Histogram=Normalized, Weighting=IDF2017.11 | 28.2 | 18.7 | 31.2 | — | — | — | — | |
| MP_INDModel Type=Interaction-Focused Matching Baselines2017.11 | 28.1 | 16.9 | 31.9 | — | — | — | — | |
| Score modelInput Representation=Dense2017.04 | 27.87 | 19.61 | 32.6 | — | — | — | — | |
| DRMM_NHxTVModel Type=Interaction-Focused Matching, Histogram=Normalized, Weighting=Term Vector2017.11 | 25.8 | 16 | 29.3 | — | — | — | — | |
| DRMM_NHxIDFModel Type=Interaction-Focused Matching, Histogram=Normalized, Weighting=IDF2017.11 | 19.9 | 14.5 | 24.3 | — | — | — | — | |
| DRMM_NHxTVModel Type=Interaction-Focused Matching, Histogram=Normalized, Weighting=Term Vector2017.11 | 18.6 | 13.2 | 21.7 | — | — | — | — | |
| DSSM_DModel Type=Representation-Focused Matching Baselines2017.11 | 17.1 | 9.5 | 20.1 | — | — | — | — | |
| MP_COSModel Type=Interaction-Focused Matching Baselines2017.11 | 16.2 | 9.4 | 19 | — | — | — | — | |
| MP_DOTModel Type=Interaction-Focused Matching Baselines2017.11 | 15.5 | 8.3 | 15.9 | — | — | — | — | |
| DSSM_DModel Type=Representation-Focused Matching Baselines2017.11 | 14.5 | 7.8 | 16.9 | — | — | — | — | |
| ARC-IIModel Type=Interaction-Focused Matching Baselines2017.11 | 12.8 | 6.7 | 14.7 | — | — | — | — | |
| CDSSM_DModel Type=Representation-Focused Matching Baselines2017.11 | 12.5 | 6.7 | 14.6 | — | — | — | — | |
| MP_INDModel Type=Interaction-Focused Matching Baselines2017.11 | 11.8 | 6.7 | 14.2 | — | — | — | — | |
| CDSSM_DModel Type=Representation-Focused Matching Baselines2017.11 | 9.3 | 5 | 11.3 | — | — | — | — | |
| MP_DOTModel Type=Interaction-Focused Matching Baselines2017.11 | 9.2 | 4.7 | 10.4 | — | — | — | — | |
| ARC-IIModel Type=Interaction-Focused Matching Baselines2017.11 | 7.4 | 4.2 | 8.6 | — | — | — | — | |
| ARC-IModel Type=Representation-Focused Matching Baselines2017.11 | 6.5 | 4.1 | 6.6 | — | — | — | — | |
| ARC-IModel Type=Representation-Focused Matching Baselines2017.11 | 4.5 | 3 | 4.7 | — | — | — | — | |
| BAGELLLM Backbone=Qwen3-14B, Ranking Protocol=PR, LLM budget per query=502026.04 | — | — | — | 55.9 | 44.4 | 16.7 | 30.7 | |
| BAGELLLM Backbone=Qwen3-14B, Ranking Protocol=ER, LLM budget per query=502026.04 | — | — | — | 57.3 | 45.7 | 17.4 | 31.7 | |
| BAGELLLM Backbone=GPT-4o, Ranking Protocol=PR, LLM budget per query=502026.04 | — | — | — | 60.4 | 45.9 | 17.6 | 30.7 | |
| BAGELLLM Backbone=GPT-4o, Ranking Protocol=ER, LLM budget per query=502026.04 | — | — | — | 62.1 | 48.7 | 18.8 | 33.2 | |
| BM25LLM Backbone=Baseline, LLM budget per query=502026.04 | — | — | — | 38.9 | 34.9 | 13.4 | 29 | |
| Cross EncoderLLM Backbone=Baseline, LLM budget per query=502026.04 | — | — | — | 46.4 | 36.1 | 14.2 | 24.9 | |
| Dense RetrieverLLM Backbone=Baseline, LLM budget per query=502026.04 | — | — | — | 39.1 | 33.2 | 12.1 | 24.9 | |
| e5-largeEnhancement=None2026.05 | — | — | — | 77.3 | — | — | — | |
| e5-largeEnhancement=Reasoning Internalizer2026.05 | — | — | — | 81.1 | — | — | — | |
| e5-largeEnhancement=CoT Reasoner2026.05 | — | — | — | 82.1 | — | — | — | |
| gte-baseEnhancement=None2026.05 | — | — | — | 77.1 | — | — | — | |
| gte-baseEnhancement=Reasoning Internalizer2026.05 | — | — | — | 80.2 | — | — | — | |
| gte-baseEnhancement=CoT Reasoner2026.05 | — | — | — | 84 | — | — | — | |
| Pointwise LLMLLM Backbone=Qwen3-14B, Ranking Protocol=PR, LLM budget per query=502026.04 | — | — | — | 52.8 | 38.2 | 15.6 | 24.9 | |
| Pointwise LLMLLM Backbone=Qwen3-14B, Ranking Protocol=ER, LLM budget per query=502026.04 | — | — | — | 53.8 | 38.6 | 15.7 | 24.9 | |
| Pointwise LLMLLM Backbone=GPT-4o, Ranking Protocol=PR, LLM budget per query=502026.04 | — | — | — | 55.5 | 39.1 | 16.3 | 24.9 | |
| Pointwise LLMLLM Backbone=GPT-4o, Ranking Protocol=ER, LLM budget per query=502026.04 | — | — | — | 57.3 | 40 | 16.4 | 24.9 | |
| qwen3-4bEnhancement=None2026.05 | — | — | — | 87 | — | — | — | |
| qwen3-4bEnhancement=Reasoning Internalizer2026.05 | — | — | — | 87.1 | — | — | — | |
| qwen3-4bEnhancement=CoT Reasoner2026.05 | — | — | — | 86.7 | — | — | — | |
| RankGPTLLM Backbone=Baseline, Ranking Method=Listwise, LLM budget per query=502026.04 | — | — | — | 50.5 | 36.8 | 14.3 | 24.9 | |
| RankGPTLLM Backbone=GPT-4o, Ranking Method=Listwise, LLM budget per query=502026.04 | — | — | — | 50.6 | 35.2 | 14 | 21.7 | |
| snowflakeEnhancement=None2026.05 | — | — | — | 24.1 | — | — | — | |
| snowflakeEnhancement=Reasoning Internalizer2026.05 | — | — | — | 42.7 | — | — | — | |
| snowflakeEnhancement=CoT Reasoner2026.05 | — | — | — | 46 | — | — | — |