Capability Calibration on MATH
0.0267Brier ScoreProbe (train on MATH)
Evaluation results
| Method | Links | |
|---|---|---|
| Probe (train on MATH)LLM=gpt-oss-20b, Cost=< 12026.02 | 0.0267 | |
| Verbalized confidenceLLM=gpt-oss-20b, Cost=L2026.02 | 0.0275 | |
| P(True)LLM=gpt-oss-20b, Cost=L2026.02 | 0.0321 | |
| Probe (train on MATH)LLM=Olmo-3-7B-Instruct, Cost=< 12026.02 | 0.0394 | |
| Probe (train on MATH)LLM=Qwen3-8B, Cost=< 12026.02 | 0.0475 | |
| Probe (train on GSM8K)LLM=gpt-oss-20b, Cost=< 12026.02 | 0.0485 | |
| Probe (train on GSM8K)LLM=Olmo-3-7B-Instruct, Cost=< 12026.02 | 0.0545 | |
| Verbalized confidenceLLM=Olmo-3-7B-Instruct, Cost=L2026.02 | 0.0557 | |
| Probe (train on GSM8K)LLM=Qwen3-8B, Cost=< 12026.02 | 0.0715 | |
| Verbalized confidenceLLM=Qwen3-8B, Cost=L2026.02 | 0.0962 | |
| P(True)LLM=Qwen3-8B, Cost=12026.02 | 0.1126 | |
| Probe (train on TriviaQA)LLM=Olmo-3-7B-Instruct, Cost=< 12026.02 | 0.1273 | |
| P(True)LLM=Olmo-3-7B-Instruct, Cost=12026.02 | 0.14 | |
| Probe (train on TriviaQA)LLM=gpt-oss-20b, Cost=< 12026.02 | 0.1593 | |
| Uniform random baselineLLM=Qwen3-8B, Cost=N/A2026.02 | 0.2781 | |
| Uniform random baselineLLM=Olmo-3-7B-Instruct, Cost=N/A2026.02 | 0.294 | |
| Uniform random baselineLLM=gpt-oss-20b, Cost=N/A2026.02 | 0.3063 | |
| Probe (train on TriviaQA)LLM=Qwen3-8B, Cost=< 12026.02 | 0.3219 |