Calibration on SQuAD
5.87ECETemp. Scaling
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Temp. ScalingModel=Olmo2-7B-Instruct, Unsupervised=false2026.01 | 5.87 | 0.1769 | |
| Temp. ScalingModel=Llama3.1-8B-Instruct, Unsupervised=false2026.01 | 9.11 | 0.1818 | |
| Temp. ScalingModel=Qwen2.5-7B-Instruct, Unsupervised=false2026.01 | 9.66 | 0.2007 | |
| BaseCal-ReEvalModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 21.31 | 0.2216 | |
| BaseCal-ProjModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 23.93 | 0.2357 | |
| VerbalizationModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 29.56 | 0.304 | |
| BaseCal-ReEvalModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 29.59 | 0.2755 | |
| CDKCBackbone=Qwen2.5-7B-Instruct, Optimization Rounds=22026.02 | 30.29 | — | |
| BaseCal-ProjModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 31.34 | 0.2816 | |
| BaseCal-ReEvalModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 32.15 | 0.298 | |
| CDKCBackbone=Qwen2.5-7B-Instruct2026.02 | 35.69 | — | |
| VerbalizationModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 36.03 | 0.3208 | |
| GRPOBackbone=Qwen2.5-7B-Instruct2026.02 | 37.92 | — | |
| Semantic EntropyModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 39.06 | 0.3558 | |
| Semantic EntropyModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 40.41 | 0.3821 | |
| BaseCal-ProjModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 40.85 | 0.3569 | |
| VanillaModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 44.59 | 0.3733 | |
| Semantic EntropyModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 46.45 | 0.4003 | |
| VerbalizationModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 48 | 0.4646 | |
| P(true)Model=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 48.22 | 0.4582 | |
| VanillaModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 52.55 | 0.4546 | |
| P(true)Model=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 55.32 | 0.5094 | |
| P(true)Model=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 56.66 | 0.565 | |
| VanillaModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 60.12 | 0.555 | |
| BARRELBackbone=Qwen2.5-7B-Instruct2026.02 | 65.13 | — | |
| Know WhatBackbone=Qwen2.5-7B-Instruct2026.02 | 67.03 | — | |
| Vanilla LLMBackbone=Qwen2.5-7B-Instruct2026.02 | 69.23 | — | |
| Vanilla SFTBackbone=Qwen2.5-7B-Instruct2026.02 | 69.84 | — | |
| LLKD-SFTBackbone=Qwen2.5-7B-Instruct2026.02 | 71.13 | — | |
| CGKEBackbone=Qwen2.5-7B-Instruct2026.02 | 71.24 | — | |
| CRew-DPOBackbone=Qwen2.5-7B-Instruct2026.02 | 75.37 | — |