Multiple-choice Question Answering on MMLU-Pro (Calibration and Ranking)
0.037Brier ScoreGUPO + SAGE
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GUPO + SAGEFramework=GUPO, Metric=SAGE2026.06 | 0.037 | 0.113 | 0.572 | 0.412 | 0.512 | 0.57 | |
| CSFTTraining=CSFT2026.06 | 0.045 | 0.12 | 0.195 | 0.347 | 0.388 | 0.38 | |
| LoVeC-DPOTraining=LoVeC-DPO2026.06 | 0.051 | 0.144 | 0.261 | 0.403 | 0.438 | 0.46 | |
| GUPO + Kernel Language EntropyFramework=GUPO, Metric=Kernel Language Entropy2026.06 | 0.068 | 0.205 | -0.01 | 0.163 | 0.167 | 0.18 | |
| CoT + DCAStrategy=Chain-of-Thought, Alignment=DCA2026.06 | 0.094 | 0.179 | 0.1 | 0.319 | 0.342 | 0.355 | |
| Direct Verbalized + DCAStrategy=Direct Verbalized, Alignment=DCA2026.06 | 0.113 | 0.222 | 0.124 | 0.356 | 0.412 | 0.445 | |
| Direct VerbalizedStrategy=Direct Verbalized2026.06 | 0.126 | 0.246 | 0.069 | 0.303 | 0.312 | 0.325 | |
| GUPO + Semantic EntropyFramework=GUPO, Metric=Semantic Entropy2026.06 | 0.214 | 0.316 | 0.09 | 0.188 | 0.183 | 0.18 | |
| CoTStrategy=Chain-of-Thought2026.06 | 0.409 | 0.59 | 0.299 | 0.291 | 0.296 | 0.34 |