Relevance Judgment Agreement on TREC-DL 2021 (test)
0.492Cohen's KappaUMBRELA
Evaluation Results
| Method | Links | |
|---|---|---|
| UMBRELAadaptation=score thresholding2026.01 | 0.492 | |
| Rank1-7Badaptation=score thresholding, size=7B2026.01 | 0.482 | |
| Rank1-14Badaptation=score thresholding, size=14B2026.01 | 0.472 | |
| Rank1-32Badaptation=score thresholding, size=32B2026.01 | 0.461 | |
| RankLLaMA-7Badaptation=score thresholding, size=7B2026.01 | 0.387 | |
| monoT5 largeadaptation=score thresholding, size=large2026.01 | 0.373 | |
| RankLLaMA-13Badaptation=score thresholding, size=13B2026.01 | 0.36 | |
| monoT5 baseadaptation=score thresholding, size=base2026.01 | 0.341 | |
| monoT5 3Badaptation=score thresholding, size=3B2026.01 | 0.273 |