Question Answering on HotpotQA (test) (Calibration and Uncertainty Metrics)
65.71AccuracyCorrectness-only
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Correctness-onlyReward Scheme=Correctness-only2026.07 | 65.71 | 0.5095 | 0.3207 | 0.3363 | 0.3346 | -0.3345 | |
| Underconfidence-1Reward Scheme=Underconfidence-12026.07 | 65.51 | 0.6046 | 0.4238 | 0.2313 | 0.1175 | -0.1175 | |
| Overconfidence-1000Reward Scheme=Overconfidence-10002026.07 | 65.48 | 0.5273 | 0.322 | 0.3328 | 0.3307 | -0.3305 | |
| Log-1/ln 202Reward Scheme=Log-1/ln 2022026.07 | 65.37 | 0.5455 | 0.4077 | 0.2461 | 0.1491 | -0.1491 | |
| Underconfidence-4Reward Scheme=Underconfidence-42026.07 | 65.29 | 0.6158 | 0.4242 | 0.2287 | 0.1093 | -0.1093 | |
| Overconfidence-4Reward Scheme=Overconfidence-42026.07 | 65.26 | 0.5725 | 0.4134 | 0.2393 | 0.1319 | -0.1319 | |
| Brier-2Reward Scheme=Brier-22026.07 | 65.07 | 0.592 | 0.4221 | 0.2286 | 0.1057 | -0.1057 | |
| Overconfidence-1Reward Scheme=Overconfidence-12026.07 | 65.07 | 0.5854 | 0.4059 | 0.2448 | 0.1563 | -0.1562 | |
| Brier-1Reward Scheme=Brier-12026.07 | 65.04 | 0.579 | 0.4144 | 0.236 | 0.1225 | -0.1225 | |
| Log-1Reward Scheme=Log-12026.07 | 64.72 | 0.642 | 0.4277 | 0.2194 | 0.0777 | -0.0776 | |
| Brier-log HybridReward Scheme=Brier-log Hybrid2026.07 | 64.39 | 0.6098 | 0.4128 | 0.2311 | 0.1081 | -0.108 | |
| Base (JSON)Format=JSON2026.07 | 44.04 | 0.585 | -0.0286 | 0.4689 | 0.473 | -0.4693 | |
| SFT (XML)Format=XML2026.07 | 43.62 | 0.6044 | -0.0208 | 0.457 | 0.4644 | -0.4634 | |
| Base (XML)Format=XML2026.07 | 4.79 | 0.5315 | -0.7309 | 0.7788 | 0.8399 | -0.8399 | |
| Log LossReward Scheme=Log Loss2026.07 | 0 | — | 0 | 0.0001 | 0.005 | -0.0049 | |
| Brier ScoreReward Scheme=Brier Score2026.07 | 0 | — | -0.0001 | 0.0001 | 0.0058 | -0.0058 |