Question Answering on HotpotQA Modified (test)
42.04AccuracyCorrectness-only
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Correctness-onlyReward Scheme=Correctness-only2026.07 | 42.04 | 0.5292 | -0.1282 | 0.5486 | 0.5508 | -0.547 | |
| Overconfidence-1Reward Scheme=Overconfidence-12026.07 | 42.04 | 0.7332 | 0.2012 | 0.2192 | 0.1182 | -0.1152 | |
| Log- 1 / ln 202Reward Scheme=Log- 1 / ln 2022026.07 | 41.86 | 0.7081 | 0.1922 | 0.2264 | 0.1038 | -0.1026 | |
| Underconfidence-1Reward Scheme=Underconfidence-12026.07 | 41.81 | 0.7519 | 0.214 | 0.2041 | 0.0794 | -0.0775 | |
| Brier-1Reward Scheme=Brier-12026.07 | 41.73 | 0.7474 | 0.2131 | 0.2043 | 0.0637 | -0.0608 | |
| Overconfidence-4Reward Scheme=Overconfidence-42026.07 | 41.66 | 0.7405 | 0.2149 | 0.2016 | 0.0223 | -0.0129 | |
| Overconfidence-1000Reward Scheme=Overconfidence-10002026.07 | 41.35 | 0.6856 | 0.1402 | 0.2734 | 0.23 | -0.229 | |
| Underconfidence-4Reward Scheme=Underconfidence-42026.07 | 41.18 | 0.7481 | 0.2088 | 0.203 | 0.0616 | -0.0565 | |
| Brier-log HybridReward Scheme=Brier-log Hybrid2026.07 | 40.07 | 0.744 | 0.1994 | 0.2013 | 0.0503 | -0.0459 | |
| Brier-2Reward Scheme=Brier-22026.07 | 28.62 | 0.89 | 0.1677 | 0.1185 | 0.0414 | -0.0414 | |
| Base (JSON)Reward Scheme=Base (JSON)2026.07 | 28.06 | 0.6559 | -0.2226 | 0.5032 | 0.5335 | -0.5317 | |
| SFT (XML)Reward Scheme=SFT (XML)2026.07 | 27.87 | 0.6472 | -0.227 | 0.5057 | 0.5347 | -0.5325 | |
| Log-1Reward Scheme=Log-12026.07 | 9.5 | 0.9681 | 0.0536 | 0.0415 | 0.0235 | -0.0235 | |
| Base (XML)Reward Scheme=Base (XML)2026.07 | 3.63 | 0.605 | -0.6437 | 0.68 | 0.7703 | -0.7703 | |
| Log LossReward Scheme=Log Loss2026.07 | 0 | — | 0 | 0.0001 | 0.005 | -0.005 | |
| Brier ScoreReward Scheme=Brier Score2026.07 | 0 | — | 0 | 0.0001 | 0.005 | -0.005 |