Question Answering on NQ ID (Accuracy, ECE, Brier, AUROC)
69.3AccuracyCAR (MSCR)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CAR (MSCR)Model=Qwen2.5-7B2026.01 | 69.3 | 0.238 | 0.255 | 0.637 | |
| CAR (Weighted Brier, λ=1/3)Model=Qwen2.5-7B2026.01 | 67.7 | 0.281 | 0.293 | 0.629 | |
| MASHModel=Qwen2.5-7B2026.01 | 67 | 0.309 | 0.315 | 0.543 | |
| CAR (Weighted Brier, λ=1)Model=Qwen2.5-7B2026.01 | 65.2 | 0.221 | 0.242 | 0.693 | |
| Vanilla Search-R1Model=Qwen2.5-7B2026.01 | 61.1 | 0.363 | 0.367 | 0.563 | |
| Temperature ScalingModel=Qwen2.5-7B2026.01 | 61.1 | 0.311 | 0.33 | 0.563 | |
| Vanilla Search-R1Model=Qwen3-4B2026.01 | 45.7 | 0.452 | 0.438 | 0.634 | |
| Temperature ScalingModel=Qwen3-4B2026.01 | 45.7 | 0.38 | 0.377 | 0.634 | |
| CAR (MSCR)Model=Qwen2.5-3B2026.01 | 45.5 | 0.303 | 0.303 | 0.699 | |
| CAR (MSCR)Model=Qwen3-4B2026.01 | 45.5 | 0.272 | 0.272 | 0.724 | |
| CAR (Weighted Brier, λ=1/3)Model=Qwen2.5-3B2026.01 | 44.5 | 0.307 | 0.313 | 0.677 | |
| CAR (Weighted Brier, λ=1/3)Model=Qwen3-4B2026.01 | 44.2 | 0.274 | 0.274 | 0.718 | |
| MASHModel=Qwen2.5-3B2026.01 | 43.4 | 0.479 | 0.47 | 0.612 | |
| Vanilla Search-R1Model=Qwen2.5-3B2026.01 | 43.1 | 0.528 | 0.519 | 0.6 | |
| Temperature ScalingModel=Qwen2.5-3B2026.01 | 43.1 | 0.5 | 0.489 | 0.6 | |
| MASHModel=Qwen3-4B2026.01 | 36.3 | 0.543 | 0.514 | 0.645 | |
| CAR (Weighted Brier, λ=1)Model=Qwen3-4B2026.01 | 28.5 | 0.115 | 0.12 | 0.918 | |
| CAR (Weighted Brier, λ=1)Model=Qwen2.5-3B2026.01 | 22.3 | 0.091 | 0.091 | 0.941 |