Multi-answer Question Answering on MAQA
0.04Hamming DistanceP(True)
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| P(True)Backbone=Qwen3-30B-A3B2026.01 | 0.04 | 0.028 | — | — | |
| P(True)Backbone=Gemma-3-4B2026.01 | 0.092 | 0.117 | — | — | |
| KLEBackbone=Qwen3-4B2026.01 | 0.237 | 0.564 | — | — | |
| MSPBackbone=Qwen3-4B2026.01 | 0.254 | 0.476 | — | — | |
| SEBackbone=Qwen3-4B2026.01 | 0.327 | 0.645 | — | — | |
| COCOABackbone=Qwen3-4B2026.01 | 0.335 | 0.57 | — | — | |
| P(True)Backbone=Qwen3-4B2026.01 | 0.347 | 0.235 | — | — | |
| MSPBackbone=Gemma-3-4B2026.01 | 0.355 | 0.534 | — | — | |
| SARBackbone=Qwen3-4B2026.01 | 0.419 | 0.611 | — | — | |
| P(True)Backbone=Gemma-3-12B2026.01 | 0.43 | 0.366 | — | — | |
| MSPBackbone=Gemma-3-12B2026.01 | 0.445 | 0.527 | — | — | |
| COCOABackbone=Gemma-3-4B2026.01 | 0.461 | 0.636 | — | — | |
| MSPBackbone=Qwen3-30B-A3B2026.01 | 0.489 | 0.545 | — | — | |
| Task CalibrationModel=Qwen3-30B-A3B, Decoding Strategy=Ours2026.05 | 0.49 | — | — | — | |
| SARBackbone=Gemma-3-4B2026.01 | 0.509 | 0.713 | — | — | |
| Task CalibrationModel=Gemma-3-12B, Decoding Strategy=Ours2026.05 | 0.51 | — | — | — | |
| MBRModel=Qwen3-30B-A3B, Decoding Strategy=MBR2026.05 | 0.54 | — | — | — | |
| KLEBackbone=Qwen3-30B-A3B2026.01 | 0.554 | 0.586 | — | — | |
| KLEBackbone=Gemma-3-4B2026.01 | 0.563 | 0.727 | — | — | |
| SEBackbone=Qwen3-30B-A3B2026.01 | 0.585 | 0.601 | — | — | |
| COCOABackbone=Gemma-3-12B2026.01 | 0.588 | 0.666 | — | — | |
| COCOABackbone=Qwen3-30B-A3B2026.01 | 0.589 | 0.627 | — | — | |
| SARBackbone=Qwen3-30B-A3B2026.01 | 0.592 | 0.624 | — | — | |
| KLEBackbone=Gemma-3-12B2026.01 | 0.599 | 0.685 | — | — | |
| MBRModel=Gemma-3-12B, Decoding Strategy=MBR2026.05 | 0.6 | — | — | — | |
| Task CalibrationModel=Gemma-3-4B, Decoding Strategy=Ours2026.05 | 0.62 | — | — | — | |
| BeamModel=Qwen3-30B-A3B, Decoding Strategy=Beam2026.05 | 0.62 | — | — | — | |
| SARBackbone=Gemma-3-12B2026.01 | 0.624 | 0.732 | — | — | |
| SEBackbone=Gemma-3-4B2026.01 | 0.629 | 0.673 | — | — | |
| AncestralModel=Qwen3-30B-A3B, Decoding Strategy=Ancestral2026.05 | 0.64 | — | — | — | |
| GreedyModel=Qwen3-30B-A3B, Decoding Strategy=Greedy2026.05 | 0.64 | — | — | — | |
| SEBackbone=Gemma-3-12B2026.01 | 0.647 | 0.675 | — | — | |
| R_BayesBackbone=Qwen3-30B-A3B2026.01 | 0.66 | 0.672 | — | — | |
| BeamModel=Gemma-3-12B, Decoding Strategy=Beam2026.05 | 0.69 | — | — | — | |
| R_BayesBackbone=Gemma-3-4B2026.01 | 0.705 | 0.753 | — | — | |
| Task CalibrationModel=Qwen-3-4B, Decoding Strategy=Ours2026.05 | 0.72 | — | — | — | |
| R_BayesBackbone=Qwen3-4B2026.01 | 0.731 | 0.805 | — | — | |
| R_BayesBackbone=Gemma-3-12B2026.01 | 0.737 | 0.741 | — | — | |
| MBRModel=Gemma-3-4B, Decoding Strategy=MBR2026.05 | 0.74 | — | — | — | |
| AncestralModel=Gemma-3-12B, Decoding Strategy=Ancestral2026.05 | 0.74 | — | — | — | |
| GreedyModel=Gemma-3-12B, Decoding Strategy=Greedy2026.05 | 0.74 | — | — | — | |
| ContrastiveModel=Qwen3-30B-A3B, Decoding Strategy=Contrastive2026.05 | 0.74 | — | — | — | |
| ContrastiveModel=Gemma-3-12B, Decoding Strategy=Contrastive2026.05 | 0.77 | — | — | — | |
| MBRModel=Qwen-3-4B, Decoding Strategy=MBR2026.05 | 0.87 | — | — | — | |
| BeamModel=Gemma-3-4B, Decoding Strategy=Beam2026.05 | 0.89 | — | — | — | |
| GreedyModel=Gemma-3-4B, Decoding Strategy=Greedy2026.05 | 0.94 | — | — | — | |
| AncestralModel=Gemma-3-4B, Decoding Strategy=Ancestral2026.05 | 0.97 | — | — | — | |
| BeamModel=Qwen-3-4B, Decoding Strategy=Beam2026.05 | 1.03 | — | — | — | |
| GreedyModel=Qwen-3-4B, Decoding Strategy=Greedy2026.05 | 1.03 | — | — | — | |
| AncestralModel=Qwen-3-4B, Decoding Strategy=Ancestral2026.05 | 1.09 | — | — | — | |
| ContrastiveModel=Qwen-3-4B, Decoding Strategy=Contrastive2026.05 | 1.2 | — | — | — | |
| ContrastiveModel=Gemma-3-4B, Decoding Strategy=Contrastive2026.05 | 1.45 | — | — | — | |
| COCOAModel=Gemma-3-4B2026.01 | — | — | 0.712 | 0.751 | |
| COCOAModel=Gemma-3-12B2026.01 | — | — | 0.756 | 0.761 | |
| COCOAModel=Qwen3-4B2026.01 | — | — | 0.717 | 0.726 | |
| COCOAModel=Qwen3-30B-A3B2026.01 | — | — | 0.748 | 0.738 | |
| KLEModel=Gemma-3-4B2026.01 | — | — | 0.762 | 0.791 | |
| KLEModel=Gemma-3-12B2026.01 | — | — | 0.771 | 0.772 | |
| KLEModel=Qwen3-4B2026.01 | — | — | 0.701 | 0.725 | |
| KLEModel=Qwen3-30B-A3B2026.01 | — | — | 0.733 | 0.719 | |
| MSPModel=Gemma-3-4B2026.01 | — | — | 0.676 | 0.71 | |
| MSPModel=Gemma-3-12B2026.01 | — | — | 0.716 | 0.719 | |
| MSPModel=Qwen3-4B2026.01 | — | — | 0.692 | 0.694 | |
| MSPModel=Qwen3-30B-A3B2026.01 | — | — | 0.718 | 0.711 | |
| P(True)Model=Gemma-3-4B2026.01 | — | — | 0.557 | 0.548 | |
| P(True)Model=Gemma-3-12B2026.01 | — | — | 0.67 | 0.654 | |
| P(True)Model=Qwen3-4B2026.01 | — | — | 0.59 | 0.568 | |
| P(True)Model=Qwen3-30B-A3B2026.01 | — | — | 0.499 | 0.505 | |
| RBayesModel=Gemma-3-4B2026.01 | — | — | 0.794 | 0.803 | |
| RBayesModel=Gemma-3-12B2026.01 | — | — | 0.81 | 0.8 | |
| RBayesModel=Qwen3-4B2026.01 | — | — | 0.771 | 0.809 | |
| RBayesModel=Qwen3-30B-A3B2026.01 | — | — | 0.764 | 0.752 | |
| SARModel=Gemma-3-4B2026.01 | — | — | 0.734 | 0.784 | |
| SARModel=Gemma-3-12B2026.01 | — | — | 0.775 | 0.789 | |
| SARModel=Qwen3-4B2026.01 | — | — | 0.721 | 0.736 | |
| SARModel=Qwen3-30B-A3B2026.01 | — | — | 0.752 | 0.743 | |
| SEModel=Gemma-3-4B2026.01 | — | — | 0.777 | 0.772 | |
| SEModel=Gemma-3-12B2026.01 | — | — | 0.768 | 0.762 | |
| SEModel=Qwen3-4B2026.01 | — | — | 0.712 | 0.746 | |
| SEModel=Qwen3-30B-A3B2026.01 | — | — | 0.742 | 0.724 |