Relevance Judgment Agreement on TREC-DL 2023 (test)
0.418Cohen's KappaUMBRELA
Evaluation Results
| Method | Links | |
|---|---|---|
| UMBRELAadaptation=score thresholding2026.01 | 0.418 | |
| Rank1-14Badaptation=score thresholding, size=14B2026.01 | 0.406 | |
| Rank1-32Badaptation=score thresholding, size=32B2026.01 | 0.392 | |
| Rank1-7Badaptation=score thresholding, size=7B2026.01 | 0.386 | |
| monoT5 3Badaptation=score thresholding, size=3B2026.01 | 0.336 | |
| RankLLaMA-13Badaptation=score thresholding, size=13B2026.01 | 0.314 | |
| RankLLaMA-7Badaptation=score thresholding, size=7B2026.01 | 0.311 | |
| monoT5 largeadaptation=score thresholding, size=large2026.01 | 0.295 | |
| monoT5 baseadaptation=score thresholding, size=base2026.01 | 0.269 |