Calibration on WebQ
0.0674ECETemp. Scaling
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Temp. ScalingModel=Olmo2-7B-Instruct, Unsupervised=false2026.01 | 0.0674 | 0.2191 | |
| Temp. ScalingModel=Qwen2.5-7B-Instruct, Unsupervised=false2026.01 | 0.0738 | 0.2341 | |
| Temp. ScalingModel=Llama3.1-8B-Instruct, Unsupervised=false2026.01 | 0.093 | 0.2446 | |
| BaseCal-ReEvalModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 0.1587 | 0.2483 | |
| BaseCal-ProjModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 0.1792 | 0.2696 | |
| BaseCal-ReEvalModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 0.1873 | 0.2737 | |
| BaseCal-ProjModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 0.1967 | 0.2652 | |
| BaseCal-ReEvalModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 0.1983 | 0.2801 | |
| BaseCal-ProjModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 0.2091 | 0.2764 | |
| Semantic EntropyModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 0.2469 | 0.2769 | |
| Semantic EntropyModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 0.2497 | 0.2853 | |
| VerbalizationModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 0.2537 | 0.3101 | |
| VerbalizationModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 0.3565 | 0.3456 | |
| Semantic EntropyModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 0.3635 | 0.3446 | |
| VanillaModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 0.3638 | 0.3514 | |
| VanillaModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 0.3832 | 0.38 | |
| VerbalizationModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 0.3977 | 0.3957 | |
| P(true)Model=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 0.4071 | 0.4043 | |
| VanillaModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 0.4486 | 0.4328 | |
| P(true)Model=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 0.4581 | 0.4496 | |
| P(true)Model=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 0.4838 | 0.4829 | |
| CDKCBackbone=Qwen2.5-7B-Instruct, Optimization Rounds=22026.02 | 14.85 | — | |
| GRPOBackbone=Qwen2.5-7B-Instruct2026.02 | 23.86 | — | |
| CDKCBackbone=Qwen2.5-7B-Instruct2026.02 | 23.93 | — | |
| BARRELBackbone=Qwen2.5-7B-Instruct2026.02 | 41.43 | — | |
| Know WhatBackbone=Qwen2.5-7B-Instruct2026.02 | 52.27 | — | |
| LLKD-SFTBackbone=Qwen2.5-7B-Instruct2026.02 | 53.51 | — | |
| Vanilla SFTBackbone=Qwen2.5-7B-Instruct2026.02 | 53.77 | — | |
| CGKEBackbone=Qwen2.5-7B-Instruct2026.02 | 54.67 | — | |
| Vanilla LLMBackbone=Qwen2.5-7B-Instruct2026.02 | 54.81 | — | |
| CRew-DPOBackbone=Qwen2.5-7B-Instruct2026.02 | 57.03 | — |