Uncertainty Estimation on SimpleQA
61.3AUROCP(YES) (top right)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| P(YES) (top right)Model=Phi-3.8B2025.06 | 61.3 | 17.8 | 18.8 | |
| Internal ConfidenceModel=Phi-3.8B2025.06 | 61.2 | 26.1 | 18.2 | |
| P(YES) (naive avg)Model=Phi-3.8B2025.06 | 59.8 | 17.8 | 69.9 | |
| P(YES) (naive avg)Model=Qwen-14B2025.06 | 57.6 | 16.6 | 4 | |
| Internal ConfidenceModel=Qwen-14B2025.06 | 56 | 13.2 | 10.3 | |
| Internal ConfidenceModel=Llama-8B2025.06 | 55.6 | 11.6 | 67.4 | |
| P(YES) (top right)Model=Qwen-14B2025.06 | 55.6 | 11.4 | 35.5 | |
| P(YES) (naive avg)Model=Llama-8B2025.06 | 54.9 | 5.8 | 28.5 | |
| Predictive EntropyModel=Phi-3.8B2025.06 | 54 | 5.9 | — | |
| Min-K EntropyModel=Phi-3.8B2025.06 | 53.8 | 13.2 | — | |
| P(YES) (top right)Model=Llama-8B2025.06 | 53.6 | 5.2 | 78.9 | |
| PerplexityModel=Phi-3.8B2025.06 | 52.4 | 5.8 | — | |
| Min-K EntropyModel=Qwen-14B2025.06 | 51.8 | 8.1 | — | |
| Max(− log p)Model=Qwen-14B2025.06 | 50.8 | -1.2 | — | |
| Max(− log p)Model=Phi-3.8B2025.06 | 50.5 | 5.5 | — | |
| Predictive EntropyModel=Qwen-14B2025.06 | 50.5 | 1.3 | — | |
| Attentional EntropyModel=Phi-3.8B2025.06 | 50.1 | 5.1 | — | |
| Max(− log p)Model=Llama-8B2025.06 | 50.1 | -2.9 | — | |
| PerplexityModel=Llama-8B2025.06 | 50.1 | -3.8 | — | |
| PerplexityModel=Qwen-14B2025.06 | 50.1 | -2.4 | — | |
| Min-K EntropyModel=Llama-8B2025.06 | 49.8 | 0.1 | — | |
| Predictive EntropyModel=Llama-8B2025.06 | 49.1 | -0.9 | — | |
| Attentional EntropyModel=Qwen-14B2025.06 | 48.7 | -2.9 | — | |
| Attentional EntropyModel=Llama-8B2025.06 | 48.6 | -4.2 | — |