LLM-as-a-Judge Calibration on SummEval (test)
0.043Test Risk (MSE)Test oracle envelope
Evaluation results
| Method | Links | |
|---|---|---|
| Test oracle envelopeCalibration Budget=Largest available2026.05 | 0.043 | |
| Information-first + val. KCalibration Budget=Largest available2026.05 | 0.044 | |
| Best single judge (val.)Calibration Budget=Largest available2026.05 | 0.045 | |
| Path-family envelopeCalibration Budget=Largest available2026.05 | 0.045 | |
| Complexity-penalized + val. KCalibration Budget=Largest available2026.05 | 0.046 | |
| Random path + val. KCalibration Budget=Largest available2026.05 | 0.048 | |
| All seven judgesCalibration Budget=Largest available2026.05 | 0.059 |