Relevance Judgment Agreement on TREC-DL 2020
0.45Cohen's KappaUMBRELA
Evaluation Results
| Method | Links | |
|---|---|---|
| UMBRELAAdaptation=Direct generation2026.01 | 0.45 | |
| Rank1-7BAdaptation=Direct generation2026.01 | 0.45 | |
| Rank1-14BAdaptation=Direct generation2026.01 | 0.433 | |
| Rank1-32BAdaptation=Direct generation2026.01 | 0.421 | |
| monoT5 3BAdaptation=Direct generation2026.01 | 0.385 | |
| monoT5 largeAdaptation=Direct generation2026.01 | 0.323 | |
| monoT5 baseAdaptation=Direct generation2026.01 | 0.322 |