Inter-Rater Reliability (Kappa/ICC) on MT-Bench
71.88KappaCalibraEval
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CalibraEvalBase Model=Qwen-72B2024.10 | 71.88 | 95.7 | 96.71 | |
| Qwen-72B (Default)Debiasing=None2024.10 | 71.35 | 90.33 | 91.16 | |
| CalibraEvalBase Model=Llama-3-8B2024.10 | 28.63 | 75.45 | 76.8 | |
| PrideBase Model=Llama-3-8B2024.10 | 27.38 | 72.27 | 74.5 | |
| DIBase Model=Llama-3-8B2024.10 | 15.93 | 59.11 | 65 | |
| Llama-3-8B (Default)Debiasing=None2024.10 | 14.36 | 60.96 | 73.08 |