LLM-as-a-Judge Performance on RewardBench (test)
2.72Std Dev (Reward)CalibraEval
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CalibraEvalBackbone=Qwen-14B2024.10 | 2.72 | 64.25 | |
| PrideBackbone=Qwen-14B2024.10 | 4.18 | 64.09 | |
| CalibraEvalBackbone=ChatGPT2024.10 | 5.51 | 67.13 | |
| CalibraEvalBackbone=Llama-3-8B2024.10 | 6.48 | 68.12 | |
| PrideBackbone=Llama-3-8B2024.10 | 7.51 | 66.54 | |
| PrideBackbone=ChatGPT2024.10 | 8.54 | 66.36 | |
| Qwen-14BBackbone=Qwen-14B, Configuration=Vanilla2024.10 | 11.63 | 63.14 | |
| Llama-3-8BBackbone=Llama-3-8B, Configuration=Vanilla2024.10 | 15.01 | 65.79 | |
| ChatGPTBackbone=ChatGPT, Configuration=Vanilla2024.10 | 16.79 | 65.27 |