Relevance Judgment Agreement on TREC-DL 2023
0.418Cohen's KappaUMBRELA
Evaluation Results
| Method | Links | |
|---|---|---|
| UMBRELAAdaptation=Direct generation2026.01 | 0.418 | |
| Rank1-14BAdaptation=Direct generation2026.01 | 0.394 | |
| Rank1-7BAdaptation=Direct generation2026.01 | 0.382 | |
| Rank1-32BAdaptation=Direct generation2026.01 | 0.38 | |
| monoT5 3BAdaptation=Direct generation2026.01 | 0.335 | |
| monoT5 largeAdaptation=Direct generation2026.01 | 0.291 | |
| monoT5 baseAdaptation=Direct generation2026.01 | 0.269 |