Model Calibration on MATH
3.3ECEVerbalized (steered)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Verbalized (steered)Model=Mistral-7B-Inst.-v0.3, Confidence Source=Verbalized (steered)2026.03 | 3.3 | 2.1 | 9.3 | |
| Verbalized (steered)Model=Llama-3.1-8B-Inst., Confidence Source=Verbalized (steered)2026.03 | 3.7 | 2.4 | 10.5 | |
| Verbalized (steered)Model=Qwen2.5-7B-Inst., Confidence Source=Verbalized (steered)2026.03 | 10.7 | 3.5 | 11.8 | |
| VerbalizedModel=Llama-3.1-8B-Inst., Confidence Source=Verbalized2026.03 | 14.9 | 8.4 | 20.2 | |
| SteerConfModel=Qwen2.5-7B-Inst., Confidence Source=SteerConf2026.03 | 19.5 | 25.9 | 41.4 | |
| SteerConfModel=Mistral-7B-Inst.-v0.3, Confidence Source=SteerConf2026.03 | 24.3 | 20.1 | 29.1 | |
| VerbalizedModel=Mistral-7B-Inst.-v0.3, Confidence Source=Verbalized2026.03 | 35.1 | 15.9 | 40 | |
| VerbalizedModel=Qwen2.5-7B-Inst., Confidence Source=Verbalized2026.03 | 36 | 16.7 | 36.9 | |
| SteerConfModel=Llama-3.1-8B-Inst., Confidence Source=SteerConf2026.03 | 46.3 | 40.6 | 53.6 | |
| Logit baselineModel=Llama-3.1-8B-Inst., Confidence Source=Logit baseline2026.03 | 68.4 | 50.5 | 68.5 | |
| Logit baselineModel=Mistral-7B-Inst.-v0.3, Confidence Source=Logit baseline2026.03 | 73.5 | 57.5 | 73.5 | |
| Logit baselineModel=Qwen2.5-7B-Inst., Confidence Source=Logit baseline2026.03 | 75 | 58.3 | 74.7 |