Knowledge Evaluation on MMLU-Redux
0.1083Brier ScoreIsotonic Regression
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Isotonic RegressionBackbone=Qwen3-8B, Method Category=Supervised Baseline, Calibration Protocol=Supervised2026.01 | 0.1083 | — | 0.0186 | -0.7245 | |
| Temperature ScalingBackbone=Qwen3-8B, Method Category=Supervised Baseline, Calibration Protocol=Supervised2026.01 | 0.1116 | — | 0.0584 | -0.7212 | |
| Histogram BinningBackbone=Qwen3-8B, Method Category=Supervised Baseline, Calibration Protocol=Supervised2026.01 | 0.1198 | — | 0.0177 | -0.713 | |
| Base Model (Q0)Backbone=Qwen3-8B, Method Category=Standard Model2026.01 | 0.1229 | 77.35 | 0.0326 | -0.6506 | |
| label-free post-processing frameworkBackbone=Qwen3-8B, Method Category=Label Free2026.01 | 0.1232 | 83.28 | 0.0659 | -0.7096 | |
| Base Model (Q0)Backbone=Ministral-3-8B, Method Category=Standard Model2026.01 | 0.1278 | 75.63 | 0.0534 | -0.6285 | |
| Instruct Model (Q1)Backbone=Qwen3-8B, Method Category=Standard Model2026.01 | 0.1337 | 83.28 | 0.125 | -0.6991 | |
| Histogram BinningBackbone=Ministral-3-8B, Method Category=Supervised Baseline, Calibration Protocol=Supervised2026.01 | 0.1494 | — | 0.0528 | -0.6427 | |
| Isotonic RegressionBackbone=Ministral-3-8B, Method Category=Supervised Baseline, Calibration Protocol=Supervised2026.01 | 0.1509 | — | 0.0465 | -0.6412 | |
| label-free post-processing frameworkBackbone=Ministral-3-8B, Method Category=Label Free2026.01 | 0.1519 | 80.52 | 0.0664 | -0.6534 | |
| Temperature ScalingBackbone=Ministral-3-8B, Method Category=Supervised Baseline, Calibration Protocol=Supervised2026.01 | 0.1571 | — | 0.0546 | -0.635 | |
| Base Model (Q0)Backbone=LLaMA-3.1-8B, Method Category=Standard Model2026.01 | 0.158 | 65.42 | 0.0176 | -0.4962 | |
| Instruct Model (Q1)Backbone=Ministral-3-8B, Method Category=Standard Model2026.01 | 0.172 | 79.21 | 0.1628 | -0.6201 | |
| Isotonic RegressionBackbone=LLaMA-3.1-8B, Method Category=Supervised Baseline, Calibration Protocol=Supervised2026.01 | 0.1807 | — | 0.0185 | -0.5366 | |
| Temperature ScalingBackbone=LLaMA-3.1-8B, Method Category=Supervised Baseline, Calibration Protocol=Supervised2026.01 | 0.1874 | — | 0.0352 | -0.5299 | |
| Histogram BinningBackbone=LLaMA-3.1-8B, Method Category=Supervised Baseline, Calibration Protocol=Supervised2026.01 | 0.1889 | — | 0.0188 | -0.5285 | |
| label-free post-processing frameworkBackbone=LLaMA-3.1-8B, Method Category=Label Free2026.01 | 0.2153 | 71.74 | 0.1679 | -0.5022 | |
| Instruct Model (Q1)Backbone=LLaMA-3.1-8B, Method Category=Standard Model2026.01 | 0.245 | 71.74 | 0.2417 | -0.4724 |