Uncertainty Quantification on CNN/DailyMail
0.745Hamming AUCStructure-Aware Minimum Bayes Risk Decoding
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Structure-Aware Minimum Bayes Risk DecodingBackbone Model=Gemma-3-4B2026.01 | 0.745 | 0.617 | 0.607 | |
| Structure-Aware Minimum Bayes Risk DecodingBackbone Model=Qwen3-30B-A3B2026.01 | 0.732 | 0.586 | 0.603 | |
| Structure-Aware Minimum Bayes Risk DecodingBackbone Model=Gemma-3-12B2026.01 | 0.709 | 0.585 | 0.66 | |
| Structure-Aware Minimum Bayes Risk DecodingBackbone Model=Qwen3-4B2026.01 | 0.672 | 0.556 | 0.592 | |
| COCOABackbone Model=Qwen3-30B-A3B2026.01 | 0.628 | 0.645 | 0.512 | |
| SARBackbone Model=Qwen3-30B-A3B2026.01 | 0.625 | 0.657 | 0.51 | |
| MSPBackbone Model=Qwen3-30B-A3B2026.01 | 0.605 | 0.592 | 0.517 | |
| SARBackbone Model=Qwen3-4B2026.01 | 0.604 | 0.563 | 0.526 | |
| SARBackbone Model=Gemma-3-4B2026.01 | 0.594 | 0.619 | 0.574 | |
| KLEBackbone Model=Qwen3-30B-A3B2026.01 | 0.589 | 0.57 | 0.502 | |
| COCOABackbone Model=Qwen3-4B2026.01 | 0.582 | 0.576 | 0.525 | |
| COCOABackbone Model=Gemma-3-4B2026.01 | 0.577 | 0.569 | 0.589 | |
| SEBackbone Model=Qwen3-4B2026.01 | 0.577 | 0.548 | 0.504 | |
| SEBackbone Model=Qwen3-30B-A3B2026.01 | 0.576 | 0.573 | 0.475 | |
| SEBackbone Model=Gemma-3-12B2026.01 | 0.572 | 0.534 | 0.501 | |
| SARBackbone Model=Gemma-3-12B2026.01 | 0.57 | 0.576 | 0.542 | |
| KLEBackbone Model=Qwen3-4B2026.01 | 0.57 | 0.555 | 0.492 | |
| SEBackbone Model=Gemma-3-4B2026.01 | 0.563 | 0.556 | 0.527 | |
| COCOABackbone Model=Gemma-3-12B2026.01 | 0.563 | 0.561 | 0.526 | |
| MSPBackbone Model=Qwen3-4B2026.01 | 0.546 | 0.55 | 0.52 | |
| KLEBackbone Model=Gemma-3-4B2026.01 | 0.539 | 0.51 | 0.51 | |
| KLEBackbone Model=Gemma-3-12B2026.01 | 0.534 | 0.518 | 0.479 | |
| MSPBackbone Model=Gemma-3-4B2026.01 | 0.526 | 0.519 | 0.574 | |
| MSPBackbone Model=Gemma-3-12B2026.01 | 0.523 | 0.543 | 0.515 | |
| P(True)Backbone Model=Qwen3-4B2026.01 | 0.517 | 0.483 | 0.539 | |
| P(True)Backbone Model=Qwen3-30B-A3B2026.01 | 0.515 | 0.514 | 0.499 | |
| P(True)Backbone Model=Gemma-3-4B2026.01 | 0.479 | 0.484 | 0.504 | |
| P(True)Backbone Model=Gemma-3-12B2026.01 | 0.432 | 0.458 | 0.474 |