Confidence Estimation on TL;DR
0.421Rank Correlation (RK)Verbalized Confidence
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Verbalized ConfidenceModel=Llama3-7B2026.05 | 0.421 | 58.05 | |
| Predictive ProbabilityModel=Llama3-7B2026.05 | 0.4094 | 58.93 | |
| Random AnnotatorModel=Llama3-7B2026.05 | 0.4012 | 60.34 | |
| Predictive ProbabilityModel=Qwen2.5-32B2026.05 | 0.4006 | 59.24 | |
| Simulated AnnotatorsModel=Llama3-7B2026.05 | 0.3975 | 60.58 | |
| Random AnnotatorModel=Qwen2.5-32B2026.05 | 0.3964 | 60.63 | |
| Simulated AnnotatorsModel=Qwen2.5-32B2026.05 | 0.3931 | 60.68 | |
| Learning Confidence (Vanilla)Model=Llama3-7B2026.05 | 0.3912 | 61.34 | |
| Learning Confidence (Vanilla)Model=Qwen2.5-32B2026.05 | 0.364 | 63.21 | |
| Learning Confidence (Ours)Model=Llama3-7B2026.05 | 0.3461 | 65.39 | |
| Learning Confidence (Ours)Model=Qwen2.5-32B2026.05 | 0.3196 | 67.6 |