Multi-answer Question Answering (Sets) on MAQA {0, 1}^K
102.3Hamming ScoreSelf-Consistency
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Self-ConsistencyModel=Qwen3-4B, Decoding Strategy=Self-Consistency2026.01 | 102.3 | 49.6 | |
| BeamModel=Qwen3-4B, Decoding Strategy=Beam2026.01 | 95.3 | 50.9 | |
| Self-ConsistencyModel=Gemma-3-4B, Decoding Strategy=Self-Consistency2026.01 | 92.2 | 56 | |
| ℓMAPModel=Qwen3-4B, Decoding Strategy=ℓMAP2026.01 | 92 | 50 | |
| ℓSampleModel=Qwen3-4B, Decoding Strategy=ℓSample2026.01 | 91.7 | 50.8 | |
| BeamModel=Gemma-3-4B, Decoding Strategy=Beam2026.01 | 89.2 | 54.8 | |
| ℓMAPModel=Gemma-3-4B, Decoding Strategy=ℓMAP2026.01 | 87.9 | 55.6 | |
| Bayes (ours)Model=Qwen3-4B, Decoding Strategy=Bayes (ours)2026.01 | 85.8 | 51.2 | |
| ℓSampleModel=Gemma-3-4B, Decoding Strategy=ℓSample2026.01 | 83.8 | 57 | |
| Bayes (ours)Model=Gemma-3-4B, Decoding Strategy=Bayes (ours)2026.01 | 72.9 | 57.6 | |
| Self-ConsistencyModel=Gemma-3-12B, Decoding Strategy=Self-Consistency2026.01 | 72.3 | 67.4 | |
| ℓMAPModel=Gemma-3-12B, Decoding Strategy=ℓMAP2026.01 | 69.1 | 67.6 | |
| BeamModel=Gemma-3-12B, Decoding Strategy=Beam2026.01 | 67.4 | 68.4 | |
| ℓSampleModel=Gemma-3-12B, Decoding Strategy=ℓSample2026.01 | 67.4 | 68 | |
| Bayes (ours)Model=Gemma-3-12B, Decoding Strategy=Bayes (ours)2026.01 | 60.1 | 68.4 | |
| Self-ConsistencyModel=Qwen3-30B-A3B, Decoding Strategy=Self-Consistency2026.01 | 59.5 | 68.5 | |
| BeamModel=Qwen3-30B-A3B, Decoding Strategy=Beam2026.01 | 58.1 | 69.5 | |
| ℓSampleModel=Qwen3-30B-A3B, Decoding Strategy=ℓSample2026.01 | 55.1 | 69.5 | |
| ℓMAPModel=Qwen3-30B-A3B, Decoding Strategy=ℓMAP2026.01 | 55 | 69.6 | |
| Bayes (ours)Model=Qwen3-30B-A3B, Decoding Strategy=Bayes (ours)2026.01 | 52.5 | 69.8 |