Relevance Judgment Agreement on TREC-DL 2019 (test)
0.499Cohen's KappaUMBRELA
Evaluation Results
| Method | Links | |
|---|---|---|
| UMBRELAadaptation=score thresholding2026.01 | 0.499 | |
| Rank1-14Badaptation=score thresholding, size=14B2026.01 | 0.463 | |
| Rank1-32Badaptation=score thresholding, size=32B2026.01 | 0.454 | |
| Rank1-7Badaptation=score thresholding, size=7B2026.01 | 0.443 | |
| RankLLaMA-13Badaptation=score thresholding, size=13B2026.01 | 0.407 | |
| RankLLaMA-7Badaptation=score thresholding, size=7B2026.01 | 0.4 | |
| monoT5 largeadaptation=score thresholding, size=large2026.01 | 0.398 | |
| monoT5 3Badaptation=score thresholding, size=3B2026.01 | 0.391 | |
| monoT5 baseadaptation=score thresholding, size=base2026.01 | 0.35 |