Confidence Estimation on Chatbot Arena
0.3524Rank Correlation (RK)Verbalized Confidence
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Verbalized ConfidenceModel=Llama3-7B2026.05 | 0.3524 | 64.71 | |
| Random AnnotatorModel=Llama3-7B2026.05 | 0.3428 | 65.47 | |
| Predictive ProbabilityModel=Llama3-7B2026.05 | 0.3407 | 66.15 | |
| Simulated AnnotatorsModel=Llama3-7B2026.05 | 0.3376 | 65.91 | |
| Learning Confidence (Vanilla)Model=Llama3-7B2026.05 | 0.2956 | 69.13 | |
| Random AnnotatorModel=Qwen2.5-32B2026.05 | 0.2812 | 72.04 | |
| Predictive ProbabilityModel=Qwen2.5-32B2026.05 | 0.2708 | 72.93 | |
| Simulated AnnotatorsModel=Qwen2.5-32B2026.05 | 0.2697 | 73.09 | |
| Learning Confidence (Ours)Model=Llama3-7B2026.05 | 0.2658 | 72.01 | |
| Learning Confidence (Vanilla)Model=Qwen2.5-32B2026.05 | 0.2574 | 74.33 | |
| Learning Confidence (Ours)Model=Qwen2.5-32B2026.05 | 0.2253 | 77.49 |