# Confidence Calibration on BeyondAIME (test) Source: [Confidence Calibration on BeyondAIME (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:03:00.906Z Evaluation of model confidence calibration performance on the test split of the BeyondAIME dataset. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Confidence Calibration | | Dataset (including any stated split) | BeyondAIME (test) | | Metric | SNR Gain | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 15 of 15 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [SNR Gain](): Higher is better; unit: SCALAR. - [Confidence AUC](): Higher is better; unit: SCALAR. - [Absolute Accuracy](): Higher is better; unit: PERCENT. - [Smoothed ECE](): Lower is better; unit: SCALAR. - [Brier Score](): Lower is better; unit: SCALAR. - [Negative Log Likelihood](): Lower is better; unit: SCALAR. - [Prediction Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | SNR Gain | Result date | Report paper | | --- | --- | --- | --- | | Qwen3-4B-Instruct-ppo-value | 1.202 | Not recorded | 694b739553fdd5c361739b4e | | Qwen3-4B-Instruct-confidence-ce | 1.097 | Not recorded | 694b739553fdd5c361739b4e | | Qwen3-4B-Instruct-confidence-prod | 0.806 | Not recorded | 694b739553fdd5c361739b4e | | Qwen3-4B-instruct-confidence-brier | 0.802 | Not recorded | 694b739553fdd5c361739b4e | | Qwen3-4B-Instruct-confidence-min | 0.683 | Not recorded | 694b739553fdd5c361739b4e | | o4-mini | 0.515 | Not recorded | 694b739553fdd5c361739b4e | | GPT-5 | 0.207 | Not recorded | 694b739553fdd5c361739b4e | | GPT-oss-120b | 0.154 | Not recorded | 694b739553fdd5c361739b4e | | Qwen3-4B-instruct-ppo | 0.101 | Not recorded | 694b739553fdd5c361739b4e | | Kimi-K2-0905 | 0.087 | Not recorded | 694b739553fdd5c361739b4e | | Qwen3-4B-Instruct | 0.084 | Not recorded | 694b739553fdd5c361739b4e | | Gemini-2.5-Pro | 0.071 | Not recorded | 694b739553fdd5c361739b4e | | Qwen3-max | 0.043 | Not recorded | 694b739553fdd5c361739b4e | | GLM-4.5 | 0.016 | Not recorded | 694b739553fdd5c361739b4e | | Qwen3-next-80b-a3b-thinking | 0.006 | Not recorded | 694b739553fdd5c361739b4e | ### Qwen3-4B-Instruct-ppo-value SNR Gain: 1.202. Result ID: 699945b25698665ec4afd579. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method":"Critic Value"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct-confidence-ce SNR Gain: 1.097. Result ID: 699945b25698665ec4afd578. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method":"Behaviorally Calibrated Reinforcement Learning","Loss Type":"ce"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct-confidence-prod SNR Gain: 0.806. Result ID: 699945b25698665ec4afd577. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method":"Behaviorally Calibrated Reinforcement Learning","Loss Type":"prod"}. This record does not assert independent reproduction. ### Qwen3-4B-instruct-confidence-brier SNR Gain: 0.802. Result ID: 699945b25698665ec4afd576. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method":"Behaviorally Calibrated Reinforcement Learning","Loss Type":"brier"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct-confidence-min SNR Gain: 0.683. Result ID: 699945b25698665ec4afd575. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method":"Behaviorally Calibrated Reinforcement Learning","Loss Type":"min"}. This record does not assert independent reproduction. ### o4-mini SNR Gain: 0.515. Result ID: 699945b25698665ec4afd574. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### GPT-5 SNR Gain: 0.207. Result ID: 699945b25698665ec4afd573. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### GPT-oss-120b SNR Gain: 0.154. Result ID: 699945b25698665ec4afd572. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen3-4B-instruct-ppo SNR Gain: 0.101. Result ID: 699945b25698665ec4afd571. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Kimi-K2-0905 SNR Gain: 0.087. Result ID: 699945b25698665ec4afd570. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen3-4B-Instruct SNR Gain: 0.084. Result ID: 699945b25698665ec4afd56f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Gemini-2.5-Pro SNR Gain: 0.071. Result ID: 699945b25698665ec4afd56e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen3-max SNR Gain: 0.043. Result ID: 699945b25698665ec4afd56d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### GLM-4.5 SNR Gain: 0.016. Result ID: 699945b25698665ec4afd56c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen3-next-80b-a3b-thinking SNR Gain: 0.006. Result ID: 699945b25698665ec4afd56b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.