System Ranking Correlation on TREC-DL 2020 1 (test)
0.93Kendall's tau (MAP@100)UMBRELA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| UMBRELAadaptation=direct generation2026.01 | 0.93 | 0.837 | |
| Rank1adaptation=direct generation, parameters=32B2026.01 | 0.889 | 0.834 | |
| Rank1adaptation=direct generation, parameters=14B2026.01 | 0.887 | 0.847 | |
| Rank1adaptation=direct generation, parameters=7B2026.01 | 0.883 | 0.821 | |
| monoT5adaptation=direct generation, size=3B2026.01 | 0.868 | 0.692 | |
| monoT5adaptation=direct generation, size=large2026.01 | 0.825 | 0.702 | |
| monoT5adaptation=direct generation, size=base2026.01 | 0.799 | 0.807 |