Confidence Calibration on BeyondAIME (test)
1.202SNR GainQwen3-4B-Instruct-ppo-value
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Qwen3-4B-Instruct-ppo-valueMethod=Critic Value2025.12 | 1.202 | 0.881 | 79.7 | 0.061 | 0.141 | 0.436 | 38.7 | |
| Qwen3-4B-Instruct-confidence-ceMethod=Behaviorally Calibrated Reinforcement Learning, Loss Type=ce2025.12 | 1.097 | 0.854 | 77.1 | 0.174 | 0.179 | 0.722 | 36.4 | |
| Qwen3-4B-Instruct-confidence-prodMethod=Behaviorally Calibrated Reinforcement Learning, Loss Type=prod2025.12 | 0.806 | 0.883 | 76.6 | 0.166 | 0.166 | 0.516 | 38.5 | |
| Qwen3-4B-instruct-confidence-brierMethod=Behaviorally Calibrated Reinforcement Learning, Loss Type=brier2025.12 | 0.802 | 0.902 | 78.2 | 0.177 | 0.161 | 0.527 | 39.5 | |
| Qwen3-4B-Instruct-confidence-minMethod=Behaviorally Calibrated Reinforcement Learning, Loss Type=min2025.12 | 0.683 | 0.863 | 70.3 | 0.22 | 0.194 | 0.606 | 38.4 | |
| o4-mini2025.12 | 0.515 | 0.738 | 69.8 | 0.164 | 0.241 | 0.845 | 52.7 | |
| GPT-52025.12 | 0.207 | 0.706 | 71.8 | 0.16 | 0.208 | 0.719 | 68 | |
| GPT-oss-120b2025.12 | 0.154 | 0.703 | 49.7 | 0.419 | 0.402 | 1.415 | 43.6 | |
| Qwen3-4B-instruct-ppo2025.12 | 0.101 | 0.786 | 38.8 | 0.56 | 0.518 | 1.873 | 36.1 | |
| Kimi-K2-09052025.12 | 0.087 | 0.678 | 38.5 | 0.575 | 0.529 | 2.004 | 35.5 | |
| Qwen3-4B-Instruct2025.12 | 0.084 | 0.715 | 33.8 | 0.632 | 0.599 | 2.316 | 29.1 | |
| Gemini-2.5-Pro2025.12 | 0.071 | 0.699 | 55.3 | 0.431 | 0.387 | 1.443 | 54.4 | |
| Qwen3-max2025.12 | 0.043 | 0.678 | 54 | 0.453 | 0.413 | 1.583 | 53.7 | |
| GLM-4.52025.12 | 0.016 | 0.519 | 53.6 | 0.44 | 0.449 | 1.971 | 53.2 | |
| Qwen3-next-80b-a3b-thinking2025.12 | 0.006 | 0.496 | 55.7 | 0.408 | 0.425 | 1.826 | 55.7 |