Question Answering on SimpleQA-verified OOD
42.2AccuracyVanilla Search-R1
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Vanilla Search-R1Model=Qwen3-4B2026.01 | 42.2 | 0.287 | 0.21 | 0.874 | |
| Temperature ScalingModel=Qwen3-4B2026.01 | 42.2 | 0.254 | 0.198 | 0.874 | |
| CAR (MSCR)Model=Qwen3-4B2026.01 | 41.8 | 0.106 | 0.106 | 0.929 | |
| MASHModel=Qwen2.5-7B2026.01 | 41.5 | 0.465 | 0.42 | 0.775 | |
| CAR (MSCR)Model=Qwen2.5-7B2026.01 | 40.9 | 0.15 | 0.156 | 0.837 | |
| Vanilla Search-R1Model=Qwen2.5-7B2026.01 | 40.7 | 0.441 | 0.398 | 0.776 | |
| Temperature ScalingModel=Qwen2.5-7B2026.01 | 40.7 | 0.385 | 0.347 | 0.776 | |
| CAR (Weighted Brier, λ=1/3)Model=Qwen2.5-7B2026.01 | 40.5 | 0.177 | 0.176 | 0.798 | |
| CAR (Weighted Brier, λ=1/3)Model=Qwen3-4B2026.01 | 40.1 | 0.127 | 0.127 | 0.912 | |
| MASHModel=Qwen3-4B2026.01 | 37.4 | 0.299 | 0.213 | 0.859 | |
| CAR (MSCR)Model=Qwen2.5-3B2026.01 | 36.6 | 0.192 | 0.191 | 0.763 | |
| CAR (Weighted Brier, λ=1/3)Model=Qwen2.5-3B2026.01 | 35.8 | 0.203 | 0.203 | 0.74 | |
| MASHModel=Qwen2.5-3B2026.01 | 35.1 | 0.589 | 0.56 | 0.696 | |
| Vanilla Search-R1Model=Qwen2.5-3B2026.01 | 34.8 | 0.61 | 0.587 | 0.702 | |
| Temperature ScalingModel=Qwen2.5-3B2026.01 | 34.8 | 0.583 | 0.549 | 0.702 | |
| CAR (Weighted Brier, λ=1)Model=Qwen3-4B2026.01 | 33.1 | 0.036 | 0.033 | 0.973 | |
| CAR (Weighted Brier, λ=1)Model=Qwen2.5-7B2026.01 | 24.4 | 0.05 | 0.053 | 0.94 | |
| CAR (Weighted Brier, λ=1)Model=Qwen2.5-3B2026.01 | 21.2 | 0.027 | 0.027 | 0.983 |