Relevance Judgment Agreement on TREC-DL 2019
0.499Cohen's KappaUMBRELA
Evaluation Results
| Method | Links | |
|---|---|---|
| UMBRELAAdaptation=Direct generation2026.01 | 0.499 | |
| Rank1-14BAdaptation=Direct generation2026.01 | 0.459 | |
| Rank1-32BAdaptation=Direct generation2026.01 | 0.452 | |
| Rank1-7BAdaptation=Direct generation2026.01 | 0.429 | |
| monoT5 largeAdaptation=Direct generation2026.01 | 0.425 | |
| monoT5 baseAdaptation=Direct generation2026.01 | 0.389 | |
| monoT5 3BAdaptation=Direct generation2026.01 | 0.386 |