List your company
ResearchFormerly WizWand

Multi-answer Question Answering on MAQA

0.04Hamming DistanceP(True)
-0.01640.36430.7451.1257Jan 29, 2026Feb 15, 2026Mar 4, 2026Mar 21, 2026Apr 7, 2026Apr 24, 2026May 11, 2026
Updated 5mo ago

Evaluation results

MethodLinks
P(True)Backbone=Qwen3-30B-A3B2026.010.040.028——
P(True)Backbone=Gemma-3-4B2026.010.0920.117——
KLEBackbone=Qwen3-4B2026.010.2370.564——
MSPBackbone=Qwen3-4B2026.010.2540.476——
SEBackbone=Qwen3-4B2026.010.3270.645——
COCOABackbone=Qwen3-4B2026.010.3350.57——
P(True)Backbone=Qwen3-4B2026.010.3470.235——
MSPBackbone=Gemma-3-4B2026.010.3550.534——
SARBackbone=Qwen3-4B2026.010.4190.611——
P(True)Backbone=Gemma-3-12B2026.010.430.366——
MSPBackbone=Gemma-3-12B2026.010.4450.527——
COCOABackbone=Gemma-3-4B2026.010.4610.636——
MSPBackbone=Qwen3-30B-A3B2026.010.4890.545——
Task CalibrationModel=Qwen3-30B-A3B, Decoding Strategy=Ours2026.050.49———
SARBackbone=Gemma-3-4B2026.010.5090.713——
Task CalibrationModel=Gemma-3-12B, Decoding Strategy=Ours2026.050.51———
MBRModel=Qwen3-30B-A3B, Decoding Strategy=MBR2026.050.54———
KLEBackbone=Qwen3-30B-A3B2026.010.5540.586——
KLEBackbone=Gemma-3-4B2026.010.5630.727——
SEBackbone=Qwen3-30B-A3B2026.010.5850.601——
COCOABackbone=Gemma-3-12B2026.010.5880.666——
COCOABackbone=Qwen3-30B-A3B2026.010.5890.627——
SARBackbone=Qwen3-30B-A3B2026.010.5920.624——
KLEBackbone=Gemma-3-12B2026.010.5990.685——
MBRModel=Gemma-3-12B, Decoding Strategy=MBR2026.050.6———
Task CalibrationModel=Gemma-3-4B, Decoding Strategy=Ours2026.050.62———
BeamModel=Qwen3-30B-A3B, Decoding Strategy=Beam2026.050.62———
SARBackbone=Gemma-3-12B2026.010.6240.732——
SEBackbone=Gemma-3-4B2026.010.6290.673——
AncestralModel=Qwen3-30B-A3B, Decoding Strategy=Ancestral2026.050.64———
GreedyModel=Qwen3-30B-A3B, Decoding Strategy=Greedy2026.050.64———
SEBackbone=Gemma-3-12B2026.010.6470.675——
R_BayesBackbone=Qwen3-30B-A3B2026.010.660.672——
BeamModel=Gemma-3-12B, Decoding Strategy=Beam2026.050.69———
R_BayesBackbone=Gemma-3-4B2026.010.7050.753——
Task CalibrationModel=Qwen-3-4B, Decoding Strategy=Ours2026.050.72———
R_BayesBackbone=Qwen3-4B2026.010.7310.805——
R_BayesBackbone=Gemma-3-12B2026.010.7370.741——
MBRModel=Gemma-3-4B, Decoding Strategy=MBR2026.050.74———
AncestralModel=Gemma-3-12B, Decoding Strategy=Ancestral2026.050.74———
GreedyModel=Gemma-3-12B, Decoding Strategy=Greedy2026.050.74———
ContrastiveModel=Qwen3-30B-A3B, Decoding Strategy=Contrastive2026.050.74———
ContrastiveModel=Gemma-3-12B, Decoding Strategy=Contrastive2026.050.77———
MBRModel=Qwen-3-4B, Decoding Strategy=MBR2026.050.87———
BeamModel=Gemma-3-4B, Decoding Strategy=Beam2026.050.89———
GreedyModel=Gemma-3-4B, Decoding Strategy=Greedy2026.050.94———
AncestralModel=Gemma-3-4B, Decoding Strategy=Ancestral2026.050.97———
BeamModel=Qwen-3-4B, Decoding Strategy=Beam2026.051.03———
GreedyModel=Qwen-3-4B, Decoding Strategy=Greedy2026.051.03———
AncestralModel=Qwen-3-4B, Decoding Strategy=Ancestral2026.051.09———
ContrastiveModel=Qwen-3-4B, Decoding Strategy=Contrastive2026.051.2———
ContrastiveModel=Gemma-3-4B, Decoding Strategy=Contrastive2026.051.45———
COCOAModel=Gemma-3-4B2026.01——0.7120.751
COCOAModel=Gemma-3-12B2026.01——0.7560.761
COCOAModel=Qwen3-4B2026.01——0.7170.726
COCOAModel=Qwen3-30B-A3B2026.01——0.7480.738
KLEModel=Gemma-3-4B2026.01——0.7620.791
KLEModel=Gemma-3-12B2026.01——0.7710.772
KLEModel=Qwen3-4B2026.01——0.7010.725
KLEModel=Qwen3-30B-A3B2026.01——0.7330.719
MSPModel=Gemma-3-4B2026.01——0.6760.71
MSPModel=Gemma-3-12B2026.01——0.7160.719
MSPModel=Qwen3-4B2026.01——0.6920.694
MSPModel=Qwen3-30B-A3B2026.01——0.7180.711
P(True)Model=Gemma-3-4B2026.01——0.5570.548
P(True)Model=Gemma-3-12B2026.01——0.670.654
P(True)Model=Qwen3-4B2026.01——0.590.568
P(True)Model=Qwen3-30B-A3B2026.01——0.4990.505
RBayesModel=Gemma-3-4B2026.01——0.7940.803
RBayesModel=Gemma-3-12B2026.01——0.810.8
RBayesModel=Qwen3-4B2026.01——0.7710.809
RBayesModel=Qwen3-30B-A3B2026.01——0.7640.752
SARModel=Gemma-3-4B2026.01——0.7340.784
SARModel=Gemma-3-12B2026.01——0.7750.789
SARModel=Qwen3-4B2026.01——0.7210.736
SARModel=Qwen3-30B-A3B2026.01——0.7520.743
SEModel=Gemma-3-4B2026.01——0.7770.772
SEModel=Gemma-3-12B2026.01——0.7680.762
SEModel=Qwen3-4B2026.01——0.7120.746
SEModel=Qwen3-30B-A3B2026.01——0.7420.724