LLM-as-a-Judge Calibration on Arena100K (test)
0.227Test Risk (MSE)Test oracle envelope
Evaluation results
| Method | Links | |
|---|---|---|
| Test oracle envelopeCalibration Budget=Largest available2026.05 | 0.227 | |
| Information-first + val. KCalibration Budget=Largest available2026.05 | 0.231 | |
| Complexity-penalized + val. KCalibration Budget=Largest available2026.05 | 0.232 | |
| Path-family envelopeCalibration Budget=Largest available2026.05 | 0.233 | |
| Best single judge (val.)Calibration Budget=Largest available2026.05 | 0.234 | |
| Random path + val. KCalibration Budget=Largest available2026.05 | 0.24 | |
| All seven judgesCalibration Budget=Largest available2026.05 | 0.245 |