Pairwise LLM Judging on MT-Bench
100CoverageVanilla
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VanillaJudge Model=Qwen-7B, alpha=0.102026.02 | 100 | 0.269 | — | |
| VanillaJudge Model=Qwen-14B, alpha=0.102026.02 | 100 | 0.248 | — | |
| VanillaJudge Model=Qwen-32B, alpha=0.102026.02 | 100 | 0.217 | — | |
| VanillaJudge Model=Llama-70B, alpha=0.102026.02 | 100 | 0.245 | — | |
| HeuristicJudge Model=Qwen-14B, alpha=0.102026.02 | 95.3 | 0.231 | — | |
| HeuristicJudge Model=Qwen-32B, alpha=0.102026.02 | 93.3 | 0.2 | — | |
| HeuristicJudge Model=Qwen-7B, alpha=0.102026.02 | 90.7 | 0.251 | — | |
| HeuristicJudge Model=Llama-70B, alpha=0.102026.02 | 80.9 | 0.184 | — | |
| NaiveJudge Model=Qwen-14B, alpha=0.102026.02 | 56.6 | 0.124 | — | |
| SCOPEJudge Model=Llama-70B, alpha=0.102026.02 | 54.3 | 0.098 | — | |
| SCOPEJudge Model=Qwen-14B, alpha=0.102026.02 | 50.4 | 0.098 | — | |
| SCOPEJudge Model=Qwen-32B, alpha=0.102026.02 | 50.2 | 0.099 | — | |
| NaiveJudge Model=Qwen-32B, alpha=0.102026.02 | 42.6 | 0.103 | — | |
| NaiveJudge Model=Llama-70B, alpha=0.102026.02 | 39.6 | 0.102 | — | |
| SCOPEJudge Model=Qwen-7B, alpha=0.102026.02 | 24.6 | 0.097 | — | |
| NaiveJudge Model=Qwen-7B, alpha=0.102026.02 | 10.2 | 0.116 | — | |
| Fine-tuned Rubric GeneratorJudge=Llama 3.1 70B, Iteration=12026.05 | — | — | 82.72 | |
| Fine-tuned Rubric GeneratorJudge=Llama 3.1 70B, Iteration=22026.05 | — | — | 82.93 | |
| Fine-tuned Rubric GeneratorJudge=Qwen3 14B, Iteration=12026.05 | — | — | 82.62 | |
| Fine-tuned Rubric GeneratorJudge=Qwen3 14B, Iteration=22026.05 | — | — | 82.87 | |
| Fine-tuned Rubric GeneratorJudge=Claude Sonnet 4, Iteration=12026.05 | — | — | 83.35 | |
| Fine-tuned Rubric GeneratorJudge=Claude Sonnet 4, Iteration=22026.05 | — | — | 83.69 | |
| Highest Training-free baselineJudge=Llama 3.1 70B2026.05 | — | — | 80.98 | |
| Highest Training-free baselineJudge=Qwen3 14B2026.05 | — | — | 80.55 | |
| Highest Training-free baselineJudge=Claude Sonnet 42026.05 | — | — | 81.72 |