Top-k Recall on Top-k Recall over oracles Synthetic Simulated
90Naive Top-k Recalllength residualization
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| length residualizationBiased judge=Mean over the biased judges2026.07 | 90 | 89 | — | — | |
| Bias-Aware Bayesian Bradley-Terry modelBiased judge=DeepSeek-chat, n=52026.07 | 84 | 100 | 0.16 | 4 | |
| Bias-Aware Bayesian Bradley-Terry modelBiased judge=Qwen-2.5 (7B), n=102026.07 | 66 | 86 | 0.2 | 12 | |
| order-swap majority voteBiased judge=Mean over the biased judges2026.07 | 62 | 61 | — | — | |
| Bias-Aware Bayesian Bradley-Terry modelBiased judge=Gemini-flash, n=32026.07 | 60 | 93 | 0.33 | 20 | |
| Bias-Aware Bayesian Bradley-Terry modelBiased judge=GPT-4o-mini, n=52026.07 | 56 | 84 | 0.28 | 12 | |
| Bias-Aware Bayesian Bradley-Terry modelBiased judge=Llama-3.1-8B, n=102026.07 | 50 | 90 | 0.4 | 34 |