Relevance Judgment Agreement on TREC-DL 2021
0.492Cohen's KappaUMBRELA
Evaluation results
| Method | Links | |
|---|---|---|
| UMBRELAAdaptation=Direct generation2026.01 | 0.492 | |
| Rank1-7BAdaptation=Direct generation2026.01 | 0.476 | |
| Rank1-14BAdaptation=Direct generation2026.01 | 0.469 | |
| Rank1-32BAdaptation=Direct generation2026.01 | 0.455 | |
| monoT5 3BAdaptation=Direct generation2026.01 | 0.402 | |
| monoT5 baseAdaptation=Direct generation2026.01 | 0.343 | |
| monoT5 largeAdaptation=Direct generation2026.01 | 0.34 |