Pairwise LLM Judging on RewardBench (Coverage and Risk)
100CoverageVanilla
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VanillaJudge Model=Qwen-7B, alpha=0.102026.02 | 100 | 0.243 | |
| VanillaJudge Model=Qwen-14B, alpha=0.102026.02 | 100 | 0.15 | |
| VanillaJudge Model=Qwen-32B, alpha=0.102026.02 | 100 | 0.12 | |
| VanillaJudge Model=Llama-70B, alpha=0.102026.02 | 100 | 0.134 | |
| SCOPEJudge Model=Qwen-32B, alpha=0.102026.02 | 98.3 | 0.098 | |
| HeuristicJudge Model=Qwen-14B, alpha=0.102026.02 | 96.8 | 0.14 | |
| HeuristicJudge Model=Qwen-32B, alpha=0.102026.02 | 96.4 | 0.103 | |
| NaiveJudge Model=Qwen-32B, alpha=0.102026.02 | 95.7 | 0.101 | |
| SCOPEJudge Model=Llama-70B, alpha=0.102026.02 | 93.5 | 0.099 | |
| HeuristicJudge Model=Qwen-7B, alpha=0.102026.02 | 89.4 | 0.211 | |
| SCOPEJudge Model=Qwen-14B, alpha=0.102026.02 | 89.2 | 0.099 | |
| NaiveJudge Model=Llama-70B, alpha=0.102026.02 | 87.2 | 0.101 | |
| HeuristicJudge Model=Llama-70B, alpha=0.102026.02 | 84.5 | 0.094 | |
| NaiveJudge Model=Qwen-14B, alpha=0.102026.02 | 82.7 | 0.101 | |
| SCOPEJudge Model=Qwen-7B, alpha=0.102026.02 | 59.4 | 0.099 | |
| NaiveJudge Model=Qwen-7B, alpha=0.102026.02 | 43 | 0.101 |