Multi-turn Calibration on TriviaQA
5.04ECE@1MTCal
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MTCalBackbone=Llama3.1-8B-Instruct2026.04 | 5.04 | 2.31 | 3.29 | 20.01 | 2.94 | |
| SLBackbone=Llama3.1-8B-Instruct2026.04 | 6.43 | 26.41 | 9.36 | 22.17 | 9.27 | |
| DCalBackbone=Llama3.1-8B-Instruct2026.04 | 6.6 | 21.29 | 8.36 | 22.55 | 8.05 | |
| MTCalBackbone=Qwen2.5-7B-Instruct2026.04 | 7.72 | 3.05 | 4.83 | 21.21 | 4.97 | |
| DCalBackbone=Qwen2.5-7B-Instruct2026.04 | 7.98 | 14.48 | 9.91 | 22.74 | 8.71 | |
| MTCalBackbone=Gemma2-9B-it2026.04 | 8.33 | 9.15 | 4.69 | 13.83 | 6.43 | |
| PSBackbone=Gemma2-9B-it2026.04 | 9.31 | 9.99 | 11.32 | 16.61 | 10.39 | |
| SLBackbone=Gemma2-9B-it2026.04 | 10.2 | 15.46 | 7.79 | 15.87 | 7.22 | |
| PSBackbone=Llama3.1-8B-Instruct2026.04 | 12.93 | 26.44 | 11.46 | 23.16 | 7.27 | |
| DCalBackbone=Gemma2-9B-it2026.04 | 14.67 | 16.42 | 15.41 | 16.04 | 15.53 | |
| SCBackbone=Qwen2.5-7B-Instruct2026.04 | 15.04 | 25.51 | 18.82 | 22.9 | 12.94 | |
| PSBackbone=Qwen2.5-7B-Instruct2026.04 | 17.22 | 28.89 | 18.63 | 25.7 | 15.42 | |
| SCBackbone=Llama3.1-8B-Instruct2026.04 | 19.77 | 28.1 | 25.52 | 28.88 | 20.67 | |
| VerbalBackbone=Llama3.1-8B-Instruct2026.04 | 21.96 | 41.72 | 25.89 | 27.79 | 22.94 | |
| P(True)Backbone=Llama3.1-8B-Instruct2026.04 | 22.87 | 32.47 | 23.29 | 25.5 | 21.55 | |
| SLBackbone=Qwen2.5-7B-Instruct2026.04 | 24.1 | 33.38 | 22.27 | 25.54 | 21.72 | |
| P(True)Backbone=Gemma2-9B-it2026.04 | 25.72 | 27.4 | 17.58 | 17.81 | 16.84 | |
| VerbalBackbone=Gemma2-9B-it2026.04 | 26.41 | 30.92 | 28.8 | 26.12 | 23.05 | |
| SCBackbone=Gemma2-9B-it2026.04 | 30.71 | 36.22 | 46.79 | 40.52 | 36.4 | |
| P(True)Backbone=Qwen2.5-7B-Instruct2026.04 | 31.38 | 38.27 | 27.34 | 28.61 | 26.17 | |
| VerbalBackbone=Qwen2.5-7B-Instruct2026.04 | 43.58 | 58.23 | 39.44 | 39.81 | 31.1 |