Confidence Calibration on BeyondAIME (test) benchmark leaderboard · SOTA2 Research

Confidence Calibration on BeyondAIME (test)

1.202SNR GainQwen3-4B-Instruct-ppo-value

Evaluation Results

Method
Qwen3-4B-Instruct-ppo-value2025.12	1.202	0.881	79.7	0.061	0.141	0.436	38.7
Qwen3-4B-Instruct-confidence-ce2025.12	1.097	0.854	77.1	0.174	0.179	0.722	36.4
Qwen3-4B-Instruct-confidence-prod2025.12	0.806	0.883	76.6	0.166	0.166	0.516	38.5
Qwen3-4B-instruct-confidence-brier2025.12	0.802	0.902	78.2	0.177	0.161	0.527	39.5
Qwen3-4B-Instruct-confidence-min2025.12	0.683	0.863	70.3	0.22	0.194	0.606	38.4
o4-mini2025.12	0.515	0.738	69.8	0.164	0.241	0.845	52.7
GPT-52025.12	0.207	0.706	71.8	0.16	0.208	0.719	68
GPT-oss-120b2025.12	0.154	0.703	49.7	0.419	0.402	1.415	43.6
Qwen3-4B-instruct-ppo2025.12	0.101	0.786	38.8	0.56	0.518	1.873	36.1
Kimi-K2-09052025.12	0.087	0.678	38.5	0.575	0.529	2.004	35.5
Qwen3-4B-Instruct2025.12	0.084	0.715	33.8	0.632	0.599	2.316	29.1
Gemini-2.5-Pro2025.12	0.071	0.699	55.3	0.431	0.387	1.443	54.4
Qwen3-max2025.12	0.043	0.678	54	0.453	0.413	1.583	53.7
GLM-4.52025.12	0.016	0.519	53.6	0.44	0.449	1.971	53.2
Qwen3-next-80b-a3b-thinking2025.12	0.006	0.496	55.7	0.408	0.425	1.826	55.7