Document Retrieval on Robust TREC 2004 (test)
51P@20BM25 + RM3 + T5-3B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| BM25 + RM3 + T5-3BBase Retrieval=BM25 + RM3, Reranker=T5-3B, Evaluation Protocol=Zero-shot transfer2020.03 | 51 | — | — | 0.384 | 0.601 | |
| BM25 + T5-3BBase Retrieval=BM25, Reranker=T5-3B, Evaluation Protocol=Zero-shot transfer2020.03 | 50.6 | — | — | 0.364 | 0.596 | |
| CEDRModel Type=BERT-based, Evaluation Protocol=Cross-validation2020.03 | 46.7 | — | — | — | 0.538 | |
| BirchModel Type=BERT-based, Evaluation Protocol=Tuned on Robust042020.03 | 46.7 | — | — | 0.369 | 0.532 | |
| BM25 + T5-baseBase Retrieval=BM25, Reranker=T5-base, Evaluation Protocol=Zero-shot transfer2020.03 | 42.5 | — | — | 0.314 | 0.51 | |
| BM25 + RM3 + T5-baseBase Retrieval=BM25 + RM3, Reranker=T5-base, Evaluation Protocol=Zero-shot transfer2020.03 | 42.4 | — | — | 0.32 | 0.503 | |
| BM25 + T5-largeBase Retrieval=BM25, Reranker=T5-large, Evaluation Protocol=Zero-shot transfer2020.03 | 41.6 | — | — | 0.296 | 0.499 | |
| BM25 + RM3 + T5-largeBase Retrieval=BM25 + RM3, Reranker=T5-large, Evaluation Protocol=Zero-shot transfer2020.03 | 41.5 | — | — | 0.304 | 0.495 | |
| BM25 + RM3Base Retrieval=BM25 + RM32020.03 | 38.2 | — | — | 0.29 | 0.441 | |
| BM25Base Retrieval=BM252020.03 | 36.3 | — | — | 0.253 | 0.424 | |
| BERT-FirstPpreprocessing=raw text, task_type=re-ranking top 1002019.05 | — | 0.444 | 0.491 | — | — | |
| BERT-MaxPpreprocessing=raw text, task_type=re-ranking top 1002019.05 | — | 0.469 | 0.529 | — | — | |
| BERT-SumPpreprocessing=raw text, task_type=re-ranking top 1002019.05 | — | 0.467 | 0.524 | — | — | |
| BOWpreprocessing=stopword removal and stemming, task_type=re-ranking top 1002019.05 | — | 0.417 | 0.409 | — | — | |
| Conv-KNRMpreprocessing=stopword removal and stemming, task_type=re-ranking top 1002019.05 | — | 0.416 | 0.406 | — | — | |
| Coor-Ascentpreprocessing=stopword removal and stemming, task_type=re-ranking top 1002019.05 | — | 0.427 | 0.441 | — | — | |
| DRMMpreprocessing=stopword removal and stemming, task_type=re-ranking top 1002019.05 | — | 0.422 | 0.412 | — | — | |
| RankSVMpreprocessing=stopword removal and stemming, task_type=re-ranking top 1002019.05 | — | 0.42 | 0.435 | — | — | |
| SDMpreprocessing=stopword removal and stemming, task_type=re-ranking top 1002019.05 | — | 0.427 | 0.427 | — | — |