Question Answering with Abstention on SimpleQA
0.2BASMBR
Evaluation Results
| Method | Links | |
|---|---|---|
| MBRModel=Gemma-3-4B, Decoding Strategy=MBR2026.05 | 0.2 | |
| MBRModel=Gemma-3-12B, Decoding Strategy=MBR2026.05 | 0.19 | |
| MBRModel=Qwen-3-4B, Decoding Strategy=MBR2026.05 | 0.18 | |
| BeamModel=Gemma-3-4B, Decoding Strategy=Beam2026.05 | 0.17 | |
| BeamModel=Gemma-3-12B, Decoding Strategy=Beam2026.05 | 0.17 | |
| GreedyModel=Gemma-3-4B, Decoding Strategy=Greedy2026.05 | 0.16 | |
| GreedyModel=Gemma-3-12B, Decoding Strategy=Greedy2026.05 | 0.16 | |
| ContrastiveModel=Gemma-3-12B, Decoding Strategy=Contrastive2026.05 | 0.15 | |
| ContrastiveModel=Gemma-3-4B, Decoding Strategy=Contrastive2026.05 | 0.14 | |
| GreedyModel=Qwen-3-4B, Decoding Strategy=Greedy2026.05 | 0.14 | |
| BeamModel=Qwen-3-4B, Decoding Strategy=Beam2026.05 | 0.13 | |
| ContrastiveModel=Qwen-3-4B, Decoding Strategy=Contrastive2026.05 | 0.11 | |
| MBRModel=Qwen3-30B-A3B, Decoding Strategy=MBR2026.05 | 0.07 | |
| GreedyModel=Qwen3-30B-A3B, Decoding Strategy=Greedy2026.05 | 0.04 | |
| BeamModel=Qwen3-30B-A3B, Decoding Strategy=Beam2026.05 | 0.03 | |
| ContrastiveModel=Qwen3-30B-A3B, Decoding Strategy=Contrastive2026.05 | 0.03 | |
| Task CalibrationModel=Qwen3-30B-A3B, Decoding Strategy=Ours2026.05 | 0.03 | |
| Task CalibrationModel=Gemma-3-4B, Decoding Strategy=Ours2026.05 | 0 | |
| Task CalibrationModel=Gemma-3-12B, Decoding Strategy=Ours2026.05 | 0 | |
| Task CalibrationModel=Qwen-3-4B, Decoding Strategy=Ours2026.05 | 0 | |
| AncestralModel=Qwen3-30B-A3B, Decoding Strategy=Ancestral2026.05 | -0.01 | |
| AncestralModel=Gemma-3-4B, Decoding Strategy=Ancestral2026.05 | -0.1 | |
| AncestralModel=Gemma-3-12B, Decoding Strategy=Ancestral2026.05 | -0.1 | |
| AncestralModel=Qwen-3-4B, Decoding Strategy=Ancestral2026.05 | -0.11 |