Judge Agreement and Model Ranking on ComparIA
0.156Kappa (κ)Gemma4-26B-A4B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Gemma4-26B-A4BJudge=Gemma4-26B-A4B2026.06 | 0.156 | 0.34 | 161.3 | 30.6 | 0.77 | 0.785 | 81 | |
| Gemma4-E4BJudge=Gemma4-E4B2026.06 | 0.134 | 0.3 | 146.2 | 32.3 | 0.717 | 0.746 | 78 | |
| GPT-OSS-120BJudge=GPT-OSS-120B2026.06 | 0.132 | 0.37 | 102 | 30.9 | 0.74 | 0.769 | 70 | |
| GPT-OSS-20BJudge=GPT-OSS-20B2026.06 | 0.132 | 0.34 | 107.9 | 33 | 0.719 | 0.748 | 69 | |
| Llama-3.3-70BJudge=Llama-3.3-70B2026.06 | 0.127 | 0.32 | 68 | 35.3 | 0.751 | 0.767 | 48 | |
| Qwen3-32BJudge=Qwen3-32B2026.06 | 0.124 | 0.13 | 62.4 | 49.7 | 0.808 | 0.577 | 20 | |
| Qwen3.5-27BJudge=Qwen3.5-27B2026.06 | 0.102 | 0.14 | 82.9 | 47.8 | 0.845 | 0.66 | 42 |