Uncertainty Estimation on GSM8K (1,000 samples, val)
0.668AUROCInternal Confidence
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Internal ConfidenceBackbone=Qwen-14B, Evaluation Shots=0-shot2025.06 | 0.668 | 28.2 | 5.7 | |
| Predictive EntropyBackbone=Qwen-14B, Evaluation Shots=0-shot2025.06 | 0.664 | 32.6 | — | |
| Predictive EntropyBackbone=Llama-8B, Evaluation Shots=0-shot2025.06 | 0.661 | 28 | — | |
| Internal ConfidenceBackbone=Llama-8B, Evaluation Shots=0-shot2025.06 | 0.657 | 34.9 | 3.1 | |
| P(YES) (naive avg)Backbone=Qwen-14B, Evaluation Shots=0-shot2025.06 | 0.64 | 32.3 | 32.4 | |
| Predictive EntropyBackbone=Phi-3.8B, Evaluation Shots=2-shot2025.06 | 0.636 | 25.7 | — | |
| Min-K EntropyBackbone=Qwen-14B, Evaluation Shots=0-shot2025.06 | 0.63 | 30.9 | — | |
| P(YES) (naive avg)Backbone=Llama-8B, Evaluation Shots=0-shot2025.06 | 0.613 | 18.5 | 33.5 | |
| Min-K EntropyBackbone=Phi-3.8B, Evaluation Shots=2-shot2025.06 | 0.604 | 17.9 | — | |
| Min-K EntropyBackbone=Llama-8B, Evaluation Shots=0-shot2025.06 | 0.575 | 13.2 | — | |
| Attentional EntropyBackbone=Llama-8B, Evaluation Shots=0-shot2025.06 | 0.561 | 13.5 | — | |
| Max(− log p)Backbone=Phi-3.8B, Evaluation Shots=2-shot2025.06 | 0.55 | 11.3 | — | |
| P(YES) (top right)Backbone=Qwen-14B, Evaluation Shots=0-shot2025.06 | 0.55 | 11.7 | 6.4 | |
| Attentional EntropyBackbone=Qwen-14B, Evaluation Shots=0-shot2025.06 | 0.549 | 3.1 | — | |
| Max(− log p)Backbone=Qwen-14B, Evaluation Shots=0-shot2025.06 | 0.543 | 13.5 | — | |
| PerplexityBackbone=Qwen-14B, Evaluation Shots=0-shot2025.06 | 0.54 | 7.3 | — | |
| Internal ConfidenceBackbone=Phi-3.8B, Evaluation Shots=2-shot2025.06 | 0.539 | 6.4 | 19.9 | |
| PerplexityBackbone=Phi-3.8B, Evaluation Shots=2-shot2025.06 | 0.536 | 6.9 | — | |
| P(YES) (top right)Backbone=Phi-3.8B, Evaluation Shots=2-shot2025.06 | 0.533 | 9.4 | 11.3 | |
| Max(− log p)Backbone=Llama-8B, Evaluation Shots=0-shot2025.06 | 0.533 | 10.4 | — | |
| PerplexityBackbone=Llama-8B, Evaluation Shots=0-shot2025.06 | 0.532 | 4.3 | — | |
| Internal Semantic SimilarityBackbone=Llama-8B, Evaluation Shots=0-shot2025.06 | 0.527 | 6.7 | 45.9 | |
| P(YES) (top right)Backbone=Llama-8B, Evaluation Shots=0-shot2025.06 | 0.526 | 5.2 | 11.9 | |
| Attentional EntropyBackbone=Phi-3.8B, Evaluation Shots=2-shot2025.06 | 0.524 | 4.4 | — | |
| P(YES) (naive avg)Backbone=Phi-3.8B, Evaluation Shots=2-shot2025.06 | 0.505 | 9.3 | 25.4 | |
| Internal Semantic SimilarityBackbone=Phi-3.8B, Evaluation Shots=2-shot2025.06 | 0.479 | -2.6 | 35.2 | |
| Internal Semantic SimilarityBackbone=Qwen-14B, Evaluation Shots=0-shot2025.06 | 0.475 | -4.6 | 33.1 |