Judge Agreement and Model Ranking on LMArena 140K
0.213Kappa (κ)Gemma4-26B-A4B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Gemma4-26B-A4BJudge=Gemma4-26B-A4B2026.06 | 0.213 | 0.6 | 77.5 | 19.2 | 0.938 | 0.945 | 75 | |
| Qwen3.5-27BJudge=Qwen3.5-27B2026.06 | 0.187 | 0.51 | 69.3 | 20 | 0.938 | 0.923 | 71 | |
| Gemma4-E4BJudge=Gemma4-E4B2026.06 | 0.177 | 0.41 | 88 | 18.8 | 0.924 | 0.921 | 79 | |
| DeepSeek-V3.2Judge=DeepSeek-V3.22026.06 | 0.173 | 0.4 | 70.9 | 17.4 | 0.929 | 0.93 | 75 | |
| GPT-OSS-120BJudge=GPT-OSS-120B2026.06 | 0.156 | 0.56 | 53.7 | 24 | 0.9 | 0.901 | 55 | |
| Llama-3.3-70BJudge=Llama-3.3-70B2026.06 | 0.131 | 0.47 | 56.7 | 31.1 | 0.754 | 0.752 | 45 | |
| GPT-OSS-20BJudge=GPT-OSS-20B2026.06 | 0.058 | 0.51 | 67.5 | 22.8 | 0.9 | 0.908 | 66 |