Multi-answer Question Answering on MAQA-ΔK−1
-0.149KL DivergenceP(True)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| P(True)Backbone=Gemma-3-4B2026.01 | -0.149 | — | |
| P(True)Backbone=Gemma-3-12B2026.01 | 0.083 | — | |
| P(True)Backbone=Qwen3-30B-A3B2026.01 | 0.125 | — | |
| P(True)Backbone=Qwen3-4B2026.01 | 0.148 | — | |
| KLEBackbone=Qwen3-4B2026.01 | 0.365 | — | |
| SEBackbone=Qwen3-4B2026.01 | 0.381 | — | |
| COCOABackbone=Qwen3-4B2026.01 | 0.411 | — | |
| SARBackbone=Qwen3-4B2026.01 | 0.416 | — | |
| MSPBackbone=Qwen3-4B2026.01 | 0.43 | — | |
| R_BayesBackbone=Qwen3-4B2026.01 | 0.467 | — | |
| MSPBackbone=Gemma-3-12B2026.01 | 0.54 | — | |
| SEBackbone=Gemma-3-12B2026.01 | 0.544 | — | |
| KLEBackbone=Gemma-3-12B2026.01 | 0.55 | — | |
| SARBackbone=Gemma-3-12B2026.01 | 0.573 | — | |
| COCOABackbone=Gemma-3-12B2026.01 | 0.573 | — | |
| MSPBackbone=Qwen3-30B-A3B2026.01 | 0.573 | — | |
| KLEBackbone=Qwen3-30B-A3B2026.01 | 0.58 | — | |
| SARBackbone=Qwen3-30B-A3B2026.01 | 0.581 | — | |
| SEBackbone=Qwen3-30B-A3B2026.01 | 0.587 | — | |
| R_BayesBackbone=Qwen3-30B-A3B2026.01 | 0.601 | — | |
| COCOABackbone=Qwen3-30B-A3B2026.01 | 0.619 | — | |
| R_BayesBackbone=Gemma-3-12B2026.01 | 0.634 | — | |
| MSPBackbone=Gemma-3-4B2026.01 | 0.635 | — | |
| COCOABackbone=Gemma-3-4B2026.01 | 0.679 | — | |
| R_BayesBackbone=Gemma-3-4B2026.01 | 0.696 | — | |
| SARBackbone=Gemma-3-4B2026.01 | 0.699 | — | |
| KLEBackbone=Gemma-3-4B2026.01 | 0.699 | — | |
| SEBackbone=Gemma-3-4B2026.01 | 0.705 | — | |
| Bayes (ours)Model=Qwen3-30B-A3B, Decoding Strategy=Bayes (ours)2026.01 | 5.032 | — | |
| Bayes (ours)Model=Gemma-3-12B, Decoding Strategy=Bayes (ours)2026.01 | 5.632 | — | |
| BeamModel=Qwen3-30B-A3B, Decoding Strategy=Beam2026.01 | 6.759 | — | |
| ℓMAPModel=Qwen3-30B-A3B, Decoding Strategy=ℓMAP2026.01 | 6.884 | — | |
| Self-ConsistencyModel=Qwen3-30B-A3B, Decoding Strategy=Self-Consistency2026.01 | 6.962 | — | |
| ℓSampleModel=Qwen3-30B-A3B, Decoding Strategy=ℓSample2026.01 | 7.283 | — | |
| BeamModel=Gemma-3-12B, Decoding Strategy=Beam2026.01 | 7.792 | — | |
| Bayes (ours)Model=Gemma-3-4B, Decoding Strategy=Bayes (ours)2026.01 | 7.94 | — | |
| ℓMAPModel=Gemma-3-12B, Decoding Strategy=ℓMAP2026.01 | 8.051 | — | |
| Self-ConsistencyModel=Gemma-3-12B, Decoding Strategy=Self-Consistency2026.01 | 8.232 | — | |
| ℓSampleModel=Gemma-3-12B, Decoding Strategy=ℓSample2026.01 | 8.42 | — | |
| Self-ConsistencyModel=Gemma-3-4B, Decoding Strategy=Self-Consistency2026.01 | 10.315 | — | |
| BeamModel=Gemma-3-4B, Decoding Strategy=Beam2026.01 | 10.33 | — | |
| Bayes (ours)Model=Qwen3-4B, Decoding Strategy=Bayes (ours)2026.01 | 10.415 | — | |
| ℓSampleModel=Gemma-3-4B, Decoding Strategy=ℓSample2026.01 | 10.514 | — | |
| ℓMAPModel=Gemma-3-4B, Decoding Strategy=ℓMAP2026.01 | 10.545 | — | |
| BeamModel=Qwen3-4B, Decoding Strategy=Beam2026.01 | 12.786 | — | |
| ℓMAPModel=Qwen3-4B, Decoding Strategy=ℓMAP2026.01 | 12.885 | — | |
| Self-ConsistencyModel=Qwen3-4B, Decoding Strategy=Self-Consistency2026.01 | 13.103 | — | |
| ℓSampleModel=Qwen3-4B, Decoding Strategy=ℓSample2026.01 | 13.145 | — | |
| COCOAModel=Gemma-3-4B2026.01 | — | 0.698 | |
| COCOAModel=Gemma-3-12B2026.01 | — | 0.689 | |
| COCOAModel=Qwen3-4B2026.01 | — | 0.639 | |
| COCOAModel=Qwen3-30B-A3B2026.01 | — | 0.705 | |
| KLEModel=Gemma-3-4B2026.01 | — | 0.708 | |
| KLEModel=Gemma-3-12B2026.01 | — | 0.701 | |
| KLEModel=Qwen3-4B2026.01 | — | 0.62 | |
| KLEModel=Qwen3-30B-A3B2026.01 | — | 0.688 | |
| MSPModel=Gemma-3-4B2026.01 | — | 0.685 | |
| MSPModel=Gemma-3-12B2026.01 | — | 0.668 | |
| MSPModel=Qwen3-4B2026.01 | — | 0.639 | |
| MSPModel=Qwen3-30B-A3B2026.01 | — | 0.681 | |
| P(True)Model=Gemma-3-4B2026.01 | — | 0.465 | |
| P(True)Model=Gemma-3-12B2026.01 | — | 0.539 | |
| P(True)Model=Qwen3-4B2026.01 | — | 0.555 | |
| P(True)Model=Qwen3-30B-A3B2026.01 | — | 0.529 | |
| RBayesModel=Gemma-3-4B2026.01 | — | 0.744 | |
| RBayesModel=Gemma-3-12B2026.01 | — | 0.753 | |
| RBayesModel=Qwen3-4B2026.01 | — | 0.639 | |
| RBayesModel=Qwen3-30B-A3B2026.01 | — | 0.709 | |
| SARModel=Gemma-3-4B2026.01 | — | 0.709 | |
| SARModel=Gemma-3-12B2026.01 | — | 0.704 | |
| SARModel=Qwen3-4B2026.01 | — | 0.639 | |
| SARModel=Qwen3-30B-A3B2026.01 | — | 0.689 | |
| SEModel=Gemma-3-4B2026.01 | — | 0.724 | |
| SEModel=Gemma-3-12B2026.01 | — | 0.708 | |
| SEModel=Qwen3-4B2026.01 | — | 0.624 | |
| SEModel=Qwen3-30B-A3B2026.01 | — | 0.7 |