Calibration on NQ
0.046ECETemp. Scaling
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Temp. ScalingModel=Llama3.1-8B-Instruct, Unsupervised=false2026.01 | 0.046 | — | 0.1743 | |
| Temp. ScalingModel=Olmo2-7B-Instruct, Unsupervised=false2026.01 | 0.0742 | — | 0.1826 | |
| Temp. ScalingModel=Qwen2.5-7B-Instruct, Unsupervised=false2026.01 | 0.1304 | — | 0.1938 | |
| VerbalizationModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 0.203 | — | 0.2478 | |
| BaseCal-ReEvalModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 0.2304 | — | 0.2312 | |
| CDKCBackbone=Qwen2.5-7B-Instruct, Optimization Rounds=22026.02 | 0.2401 | — | — | |
| BaseCal-ReEvalModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 0.2462 | — | 0.2349 | |
| BaseCal-ProjModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 0.2488 | — | 0.2358 | |
| VerbalizationModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 0.2689 | — | 0.2507 | |
| BaseCal-ProjModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 0.2712 | — | 0.2511 | |
| NAACLBackbone=Llama-3.1-8B-Instruct2026.01 | 0.301 | 0.724 | — | |
| P(true)Model=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 0.312 | — | 0.3166 | |
| CDKCBackbone=Qwen2.5-7B-Instruct2026.02 | 0.3132 | — | — | |
| BaseCal-ReEvalModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 0.3161 | — | 0.2714 | |
| NAACLBackbone=Qwen2.5-7B-Instruct2026.01 | 0.322 | 0.754 | — | |
| GRPOBackbone=Qwen2.5-7B-Instruct2026.02 | 0.3283 | — | — | |
| BaseCal-ProjModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 0.3382 | — | 0.288 | |
| Noise-awareBackbone=Llama-3.1-8B-Instruct2026.01 | 0.351 | 0.618 | — | |
| NAACLBackbone=DeepSeek-R1-Distill-Llama-8B2026.01 | 0.36 | 0.656 | — | |
| P(true)Model=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 0.3723 | — | 0.3708 | |
| Noise-awareBackbone=Qwen2.5-7B-Instruct2026.01 | 0.38 | 0.649 | — | |
| NAACLBackbone=DeepSeek-R1-Distill-Qwen-7B2026.01 | 0.409 | 0.726 | — | |
| CoTBackbone=Qwen2.5-7B-Instruct2026.01 | 0.414 | 0.703 | — | |
| Semantic EntropyModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 0.4178 | — | 0.3646 | |
| CoTBackbone=Llama-3.1-8B-Instruct2026.01 | 0.423 | 0.611 | — | |
| Label-only SFTBackbone=Qwen2.5-7B-Instruct2026.01 | 0.425 | 0.691 | — | |
| Noise-awareBackbone=DeepSeek-R1-Distill-Llama-8B2026.01 | 0.425 | 0.584 | — | |
| EnsembleBackbone=Llama-3.1-8B-Instruct2026.01 | 0.428 | 0.619 | — | |
| VanillaBackbone=Qwen2.5-7B-Instruct2026.01 | 0.438 | 0.71 | — | |
| VanillaModel=Olmo2-7B-Instruct, Unsupervised=true2026.01 | 0.4404 | — | 0.3739 | |
| Label-only SFTBackbone=Llama-3.1-8B-Instruct2026.01 | 0.441 | 0.658 | — | |
| P(true)Model=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 0.4439 | — | 0.3972 | |
| EnsembleBackbone=Qwen2.5-7B-Instruct2026.01 | 0.451 | 0.693 | — | |
| VanillaModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 0.4532 | — | 0.3882 | |
| VanillaBackbone=Llama-3.1-8B-Instruct2026.01 | 0.465 | 0.577 | — | |
| VerbalizationModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 0.4718 | — | 0.4623 | |
| Semantic EntropyModel=Llama3.1-8B-Instruct, Unsupervised=true2026.01 | 0.4927 | — | 0.4371 | |
| Label-only SFTBackbone=DeepSeek-R1-Distill-Llama-8B2026.01 | 0.493 | 0.588 | — | |
| VanillaBackbone=DeepSeek-R1-Distill-Llama-8B2026.01 | 0.504 | 0.637 | — | |
| Noise-awareBackbone=DeepSeek-R1-Distill-Qwen-7B2026.01 | 0.505 | 0.535 | — | |
| Semantic EntropyModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 0.5192 | — | 0.4502 | |
| VanillaBackbone=DeepSeek-R1-Distill-Qwen-7B2026.01 | 0.522 | 0.668 | — | |
| EnsembleBackbone=DeepSeek-R1-Distill-Llama-8B2026.01 | 0.525 | 0.592 | — | |
| CoTBackbone=DeepSeek-R1-Distill-Llama-8B2026.01 | 0.531 | 0.639 | — | |
| VanillaModel=Qwen2.5-7B-Instruct, Unsupervised=true2026.01 | 0.5562 | — | 0.4839 | |
| EnsembleBackbone=DeepSeek-R1-Distill-Qwen-7B2026.01 | 0.561 | 0.616 | — | |
| BARRELBackbone=Qwen2.5-7B-Instruct2026.02 | 0.561 | — | — | |
| CoTBackbone=DeepSeek-R1-Distill-Qwen-7B2026.01 | 0.578 | 0.641 | — | |
| Know WhatBackbone=Qwen2.5-7B-Instruct2026.02 | 0.6171 | — | — | |
| Label-only SFTBackbone=DeepSeek-R1-Distill-Qwen-7B2026.01 | 0.629 | 0.585 | — | |
| Vanilla SFTBackbone=Qwen2.5-7B-Instruct2026.02 | 0.6315 | — | — | |
| LLKD-SFTBackbone=Qwen2.5-7B-Instruct2026.02 | 0.6408 | — | — | |
| Vanilla LLMBackbone=Qwen2.5-7B-Instruct2026.02 | 0.6411 | — | — | |
| CGKEBackbone=Qwen2.5-7B-Instruct2026.02 | 0.6423 | — | — | |
| CRew-DPOBackbone=Qwen2.5-7B-Instruct2026.02 | 0.6782 | — | — | |
| Clustered Self-AssessmentBackbone=Qwen2.5-7B2026.06 | — | — | 0.1706 | |
| Clustered Self-AssessmentBackbone=Qwen2.5-14B2026.06 | — | — | 0.1729 | |
| Clustered Self-AssessmentBackbone=Qwen2.5-32B2026.06 | — | — | 0.1597 | |
| Clustered Self-AssessmentBackbone=Qwen3-14B2026.06 | — | — | 0.1611 | |
| Clustered Self-AssessmentBackbone=Qwen3-30B-A3B2026.06 | — | — | 0.1656 | |
| Clustered Self-AssessmentBackbone=Gemma-3-12B2026.06 | — | — | 0.1809 | |
| Clustered Self-AssessmentBackbone=Gemma-3-27B2026.06 | — | — | 0.1736 | |
| NSEBackbone=Qwen2.5-7B2026.06 | — | — | 0.2007 | |
| NSEBackbone=Qwen2.5-14B2026.06 | — | — | 0.2205 | |
| NSEBackbone=Qwen2.5-32B2026.06 | — | — | 0.1993 | |
| NSEBackbone=Qwen3-14B2026.06 | — | — | 0.193 | |
| NSEBackbone=Qwen3-30B-A3B2026.06 | — | — | 0.2011 | |
| NSEBackbone=Gemma-3-12B2026.06 | — | — | 0.2581 | |
| NSEBackbone=Gemma-3-27B2026.06 | — | — | 0.2462 | |
| P(True)Backbone=Qwen2.5-7B2026.06 | — | — | 0.1993 | |
| P(True)Backbone=Qwen2.5-14B2026.06 | — | — | 0.2128 | |
| P(True)Backbone=Qwen2.5-32B2026.06 | — | — | 0.1918 | |
| P(True)Backbone=Qwen3-14B2026.06 | — | — | 0.2015 | |
| P(True)Backbone=Qwen3-30B-A3B2026.06 | — | — | 0.2191 | |
| P(True)Backbone=Gemma-3-12B2026.06 | — | — | 0.2462 | |
| P(True)Backbone=Gemma-3-27B2026.06 | — | — | 0.2354 | |
| ProbabilityBackbone=Qwen2.5-7B2026.06 | — | — | 0.2471 | |
| ProbabilityBackbone=Qwen2.5-14B2026.06 | — | — | 0.3409 | |
| ProbabilityBackbone=Qwen2.5-32B2026.06 | — | — | 0.3155 | |
| ProbabilityBackbone=Qwen3-14B2026.06 | — | — | 0.3149 | |
| ProbabilityBackbone=Qwen3-30B-A3B2026.06 | — | — | 0.3208 | |
| ProbabilityBackbone=Gemma-3-12B2026.06 | — | — | 0.3207 | |
| ProbabilityBackbone=Gemma-3-27B2026.06 | — | — | 0.3503 |