Relevance Assessment Label Alignment on Deep Learning 2020
0.46Cohen's Kappa (κ)query-contrastive
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| query-contrastiveModel=gpt-oss-120b, q=1, d+=5, d-=52026.04 | 0.46 | 0.45 | 48 | 39 | 10 | 4 | |
| queryModel=gpt-oss-120b, q=12026.04 | 0.4 | 0.48 | 48 | 42 | 8 | 3 | |
| TREC Topic (Baseline)Model=TREC Human Judgments2026.04 | 0.39 | 0.61 | 39 | 39 | 9 | 12 |