System ranking correlation on TREC-DL 2021 2 (test)
0.921Kendall's tau (MAP@100)UMBRELA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| UMBRELAadaptation=direct generation2026.01 | 0.921 | 0.868 | |
| monoT5adaptation=direct generation, size=3B2026.01 | 0.906 | 0.828 | |
| Rank1adaptation=direct generation, parameters=7B2026.01 | 0.892 | 0.839 | |
| Rank1adaptation=direct generation, parameters=14B2026.01 | 0.89 | 0.829 | |
| Rank1adaptation=direct generation, parameters=32B2026.01 | 0.886 | 0.836 | |
| monoT5adaptation=direct generation, size=large2026.01 | 0.863 | 0.786 | |
| monoT5adaptation=direct generation, size=base2026.01 | 0.858 | 0.731 |