Question Answering on HotpotQA (Acc, ECE, Brier, AUROC)
56.8AccuracyCAR (MSCR)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CAR (MSCR)Model=Qwen2.5-7B2026.01 | 56.8 | 0.326 | 0.348 | 0.641 | |
| MASHModel=Qwen2.5-7B2026.01 | 55.6 | 0.421 | 0.422 | 0.521 | |
| Vanilla Search-R1Model=Qwen2.5-7B2026.01 | 54.8 | 0.424 | 0.423 | 0.551 | |
| Temperature ScalingModel=Qwen2.5-7B2026.01 | 54.8 | 0.371 | 0.38 | 0.551 | |
| CAR (Weighted Brier, λ=1/3)Model=Qwen2.5-7B2026.01 | 52.8 | 0.368 | 0.379 | 0.6 | |
| CAR (Weighted Brier, λ=1)Model=Qwen2.5-7B2026.01 | 48.7 | 0.33 | 0.358 | 0.583 | |
| Vanilla Search-R1Model=Qwen3-4B2026.01 | 46.6 | 0.408 | 0.391 | 0.671 | |
| Temperature ScalingModel=Qwen3-4B2026.01 | 46.6 | 0.343 | 0.34 | 0.671 | |
| CAR (MSCR)Model=Qwen3-4B2026.01 | 45.9 | 0.269 | 0.27 | 0.74 | |
| CAR (Weighted Brier, λ=1/3)Model=Qwen3-4B2026.01 | 45 | 0.281 | 0.281 | 0.727 | |
| CAR (Weighted Brier, λ=1)Model=Qwen3-4B2026.01 | 36.6 | 0.184 | 0.186 | 0.856 | |
| MASHModel=Qwen3-4B2026.01 | 35.5 | 0.498 | 0.457 | 0.695 | |
| CAR (MSCR)Model=Qwen2.5-3B2026.01 | 29.2 | 0.286 | 0.289 | 0.688 | |
| MASHModel=Qwen2.5-3B2026.01 | 28.9 | 0.656 | 0.631 | 0.598 | |
| CAR (Weighted Brier, λ=1/3)Model=Qwen2.5-3B2026.01 | 28.5 | 0.329 | 0.329 | 0.651 | |
| Vanilla Search-R1Model=Qwen2.5-3B2026.01 | 27.4 | 0.699 | 0.686 | 0.599 | |
| Temperature ScalingModel=Qwen2.5-3B2026.01 | 27.4 | 0.674 | 0.651 | 0.599 | |
| CAR (Weighted Brier, λ=1)Model=Qwen2.5-3B2026.01 | 24.3 | 0.148 | 0.148 | 0.902 |