Confidence Estimation on AlpacaEval
0.4395Rank Correlation (RK)Verbalized Confidence
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Verbalized ConfidenceModel=Llama3-7B2026.05 | 0.4395 | 0.5589 | |
| Predictive ProbabilityModel=Llama3-7B2026.05 | 0.4339 | 0.557 | |
| Random AnnotatorModel=Llama3-7B2026.05 | 0.4269 | 0.574 | |
| Predictive ProbabilityModel=Qwen2.5-32B2026.05 | 0.4159 | 0.585 | |
| Simulated AnnotatorsModel=Llama3-7B2026.05 | 0.4129 | 0.5904 | |
| Random AnnotatorModel=Qwen2.5-32B2026.05 | 0.3958 | 0.6022 | |
| Simulated AnnotatorsModel=Qwen2.5-32B2026.05 | 0.3874 | 0.6107 | |
| Learning Confidence (Vanilla)Model=Llama3-7B2026.05 | 0.3871 | 0.6189 | |
| Learning Confidence (Vanilla)Model=Qwen2.5-32B2026.05 | 0.3345 | 0.6679 | |
| Learning Confidence (Ours)Model=Llama3-7B2026.05 | 0.3297 | 0.671 | |
| Learning Confidence (Ours)Model=Qwen2.5-32B2026.05 | 0.2845 | 0.7012 |