LLM-as-a-Judge Evaluation Consistency on PreferenceBench
79.73KappaCalibraEval
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CalibraEvalBase Model=GPT4o2024.10 | 79.73 | 97.29 | 97.6 | |
| GPT4o (Default)Debiasing=None2024.10 | 79.42 | 93.5 | 94.11 | |
| CalibraEvalBase Model=Llama-3-8B2024.10 | 58.54 | 88.17 | 89.43 | |
| Llama-3-8B (Default)Debiasing=None2024.10 | 58.25 | 86.23 | 86.61 |