Calibration on TruthfulQA
2.132GainS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SEvaluation Metric=ECE'2026.04 | 2.132 | 4 | — | — | — | |
| SEvaluation Metric=GECE'2026.04 | 2.073 | 3 | — | — | — | |
| BEvaluation Metric=GECE'2026.04 | 2.014 | 0 | — | — | — | |
| GIRBEvaluation Metric=GECE'2026.04 | 2.01 | 1 | — | — | — | |
| QABEvaluation Metric=GECE'2026.04 | 1.984 | 0 | — | — | — | |
| BEvaluation Metric=ECE'2026.04 | 1.936 | 0 | — | — | — | |
| QABEvaluation Metric=ECE'2026.04 | 1.901 | 0 | — | — | — | |
| GIRBEvaluation Metric=ECE'2026.04 | 1.896 | 0 | — | — | — | |
| SEvaluation Metric=Brier'2026.04 | 1.48 | 2 | — | — | — | |
| BEvaluation Metric=Brier'2026.04 | 1.464 | 0 | — | — | — | |
| GIRBEvaluation Metric=Brier'2026.04 | 1.433 | 2 | — | — | — | |
| QABEvaluation Metric=Brier'2026.04 | 1.416 | 0 | — | — | — | |
| S-BEvaluation Metric=ECE'2026.04 | 1.008 | 0 | — | — | — | |
| HS-QABEvaluation Metric=GECE'2026.04 | 0.994 | 0 | — | — | — | |
| S-BEvaluation Metric=GECE'2026.04 | 0.981 | 0 | — | — | — | |
| HS-QABEvaluation Metric=ECE'2026.04 | 0.974 | 0 | — | — | — | |
| S-QABEvaluation Metric=GECE'2026.04 | 0.954 | 0 | — | — | — | |
| S-QABEvaluation Metric=ECE'2026.04 | 0.924 | 0 | — | — | — | |
| S-BEvaluation Metric=Brier'2026.04 | 0.886 | 0 | — | — | — | |
| HS-QABEvaluation Metric=Brier'2026.04 | 0.796 | 0 | — | — | — | |
| S-QABEvaluation Metric=Brier'2026.04 | 0.776 | 0 | — | — | — | |
| GIRBEvaluation Metric=AUAC2026.04 | 0.226 | 1 | — | — | — | |
| QABEvaluation Metric=AUAC2026.04 | 0.11 | 1 | — | — | — | |
| S-QABEvaluation Metric=AUAC2026.04 | 0.076 | 0 | — | — | — | |
| HS-QABEvaluation Metric=AUAC2026.04 | 0.053 | 1 | — | — | — | |
| NoneEvaluation Metric=ECE'2026.04 | 0 | 0 | — | — | — | |
| NoneEvaluation Metric=GECE'2026.04 | 0 | 0 | — | — | — | |
| NoneEvaluation Metric=Brier'2026.04 | 0 | 0 | — | — | — | |
| NoneEvaluation Metric=AUAC2026.04 | 0 | 1 | — | — | — | |
| S-BEvaluation Metric=AUAC2026.04 | -0.554 | 0 | — | — | — | |
| BEvaluation Metric=AUAC2026.04 | -0.651 | 0 | — | — | — | |
| SEvaluation Metric=AUAC2026.04 | -0.909 | 0 | — | — | — | |
| BaseModel=LLAMA3-8B, Temperature Scaling=false2026.05 | — | — | — | — | 46.2 | |
| BaseModel=LLAMA3-8B, Temperature Scaling=true2026.05 | — | — | — | — | 24.5 | |
| BaseModel=QWEN2-7B, Temperature Scaling=false2026.05 | — | — | — | — | 47.5 | |
| BaseModel=QWEN2-7B, Temperature Scaling=true2026.05 | — | — | — | — | 47.5 | |
| CAGE-CALType=Trained2026.05 | — | — | 6.5 | 79.66 | — | |
| Collab. Cal.Type=LLM-elicited2026.05 | — | — | 52.87 | 78.24 | — | |
| DiscoUQ-LLMType=Trained2026.05 | — | — | 6.99 | 71.93 | — | |
| GraphCalType=Trained2026.05 | — | — | 33.66 | 63.75 | — | |
| LLM-Cal (+topo)Type=LLM-elicited, Topology=Included2026.05 | — | — | 54.52 | 78.48 | — | |
| LLM-Cal (no topo)Type=LLM-elicited, Topology=None2026.05 | — | — | 54.88 | 78.5 | — | |
| Plurality shareType=Post-hoc plurality2026.05 | — | — | 25.23 | 71.16 | — | |
| Plurality share + IsotonicType=Post-hoc plurality, Technique=Isotonic regression2026.05 | — | — | 8.47 | 70.45 | — | |
| Plurality share + PlattType=Post-hoc plurality, Technique=Platt scaling2026.05 | — | — | 9.82 | 71.16 | — | |
| Plurality share + Scaling-bin.Type=Post-hoc plurality, Technique=Scaling-binning2026.05 | — | — | 8.41 | 70.83 | — | |
| PPT (Distribution)Model=LLAMA3-8B, Temperature Scaling=false2026.05 | — | — | — | — | 27.5 | |
| PPT (Distribution)Model=LLAMA3-8B, Temperature Scaling=true2026.05 | — | — | — | — | 22.3 | |
| PPT (Distribution)Model=QWEN2-7B, Temperature Scaling=false2026.05 | — | — | — | — | 38.1 | |
| PPT (Distribution)Model=QWEN2-7B, Temperature Scaling=true2026.05 | — | — | — | — | 38.3 | |
| PPT (Mean)Model=LLAMA3-8B, Temperature Scaling=false2026.05 | — | — | — | — | 37.7 | |
| PPT (Mean)Model=QWEN2-7B, Temperature Scaling=false2026.05 | — | — | — | — | 47.5 | |
| Scalar + GBTType=Trained2026.05 | — | — | 10.28 | 66.81 | — | |
| VerbalizedModel=LLAMA3-8B, Temperature Scaling=false2026.05 | — | — | — | — | 47 |