Pairwise LLM Judging on Chatbot Arena
100CoverageVanilla
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VanillaJudge Model=Qwen-7B, alpha=0.102026.02 | 100 | 0.249 | |
| VanillaJudge Model=Qwen-14B, alpha=0.102026.02 | 100 | 0.221 | |
| VanillaJudge Model=Qwen-32B, alpha=0.102026.02 | 100 | 0.219 | |
| VanillaJudge Model=Llama-70B, alpha=0.102026.02 | 100 | 0.198 | |
| HeuristicJudge Model=Qwen-14B, alpha=0.102026.02 | 95.8 | 0.204 | |
| HeuristicJudge Model=Qwen-32B, alpha=0.102026.02 | 93.7 | 0.201 | |
| HeuristicJudge Model=Qwen-7B, alpha=0.102026.02 | 90.3 | 0.224 | |
| HeuristicJudge Model=Llama-70B, alpha=0.102026.02 | 81.5 | 0.145 | |
| SCOPEJudge Model=Llama-70B, alpha=0.102026.02 | 58.3 | 0.099 | |
| NaiveJudge Model=Llama-70B, alpha=0.102026.02 | 56.5 | 0.101 | |
| SCOPEJudge Model=Qwen-32B, alpha=0.102026.02 | 55.1 | 0.099 | |
| NaiveJudge Model=Qwen-32B, alpha=0.102026.02 | 53.2 | 0.102 | |
| NaiveJudge Model=Qwen-14B, alpha=0.102026.02 | 53 | 0.114 | |
| SCOPEJudge Model=Qwen-14B, alpha=0.102026.02 | 46.6 | 0.097 | |
| SCOPEJudge Model=Qwen-7B, alpha=0.102026.02 | 43 | 0.099 | |
| NaiveJudge Model=Qwen-7B, alpha=0.102026.02 | 29.7 | 0.103 |