List your company
ResearchFormerly WizWand

Question Answering with Abstention on SimpleQA

0.2BASMBR
-0.1224-0.03870.0450.1287May 11, 2026
Updated 5mo ago

Evaluation results

MethodLinks
MBRModel=Gemma-3-4B, Decoding Strategy=MBR2026.050.2
MBRModel=Gemma-3-12B, Decoding Strategy=MBR2026.050.19
MBRModel=Qwen-3-4B, Decoding Strategy=MBR2026.050.18
BeamModel=Gemma-3-4B, Decoding Strategy=Beam2026.050.17
BeamModel=Gemma-3-12B, Decoding Strategy=Beam2026.050.17
GreedyModel=Gemma-3-4B, Decoding Strategy=Greedy2026.050.16
GreedyModel=Gemma-3-12B, Decoding Strategy=Greedy2026.050.16
ContrastiveModel=Gemma-3-12B, Decoding Strategy=Contrastive2026.050.15
ContrastiveModel=Gemma-3-4B, Decoding Strategy=Contrastive2026.050.14
GreedyModel=Qwen-3-4B, Decoding Strategy=Greedy2026.050.14
BeamModel=Qwen-3-4B, Decoding Strategy=Beam2026.050.13
ContrastiveModel=Qwen-3-4B, Decoding Strategy=Contrastive2026.050.11
MBRModel=Qwen3-30B-A3B, Decoding Strategy=MBR2026.050.07
GreedyModel=Qwen3-30B-A3B, Decoding Strategy=Greedy2026.050.04
BeamModel=Qwen3-30B-A3B, Decoding Strategy=Beam2026.050.03
ContrastiveModel=Qwen3-30B-A3B, Decoding Strategy=Contrastive2026.050.03
Task CalibrationModel=Qwen3-30B-A3B, Decoding Strategy=Ours2026.050.03
Task CalibrationModel=Gemma-3-4B, Decoding Strategy=Ours2026.050
Task CalibrationModel=Gemma-3-12B, Decoding Strategy=Ours2026.050
Task CalibrationModel=Qwen-3-4B, Decoding Strategy=Ours2026.050
AncestralModel=Qwen3-30B-A3B, Decoding Strategy=Ancestral2026.05-0.01
AncestralModel=Gemma-3-4B, Decoding Strategy=Ancestral2026.05-0.1
AncestralModel=Gemma-3-12B, Decoding Strategy=Ancestral2026.05-0.1
AncestralModel=Qwen-3-4B, Decoding Strategy=Ancestral2026.05-0.11