Classification on Yahoo
3.53Expected Calibration Error (ECE)CaliDist_verbalized (As.)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| CaliDist_verbalized (As.)LLM=PHI-4-MINI 3.8B2026.06 | 3.53 | 20.76 | |
| CaliDist (As.)LLM=Phi-4-mini (3.8B)2026.06 | 4.25 | 19.69 | |
| CaliDist_verbalized (Pr.)LLM=PHI-4-MINI 3.8B2026.06 | 4.58 | 20.85 | |
| CaliDist (Pr.)LLM=Phi-4-mini (3.8B)2026.06 | 5.58 | 18.84 | |
| CaliDist_verbalized (Sa.)LLM=PHI-4-MINI 3.8B2026.06 | 5.63 | 21.07 | |
| CaliDist (Sa.)LLM=Gemma-3 (4B)2026.06 | 5.74 | 20.57 | |
| CaliDist (Pr.)LLM=Gemma-3 (4B)2026.06 | 5.82 | 19.93 | |
| CaliDist (As.)LLM=Gemma-3 (4B)2026.06 | 6.8 | 21.11 | |
| CaliDist_verbalized (Sa.)LLM=GEMMA-3 4B2026.06 | 7.56 | 20.53 | |
| CaliDist (Sa.)LLM=Phi-4-mini (3.8B)2026.06 | 7.79 | 18.94 | |
| Temperature ScalingLLM=Phi-4-mini (3.8B)2026.06 | 8.83 | 18.57 | |
| CaliDist_verbalized (As.)LLM=GEMMA-3 4B2026.06 | 8.85 | 20.83 | |
| VerbalizedLLM=PHI-4-MINI 3.8B2026.06 | 10.64 | 21.95 | |
| CaliDist_verbalized (Pr.)LLM=GEMMA-3 4B2026.06 | 13.49 | 20.8 | |
| EntropyLLM=Phi-4-mini (3.8B)2026.06 | 17.2 | 21.68 | |
| FSDLLM=Phi-4-mini (3.8B)2026.06 | 18.45 | 21.79 | |
| VanillaLLM=Phi-4-mini (3.8B)2026.06 | 20.43 | 22.72 | |
| VerbalizedLLM=GEMMA-3 4B2026.06 | 23.11 | 34.03 | |
| ConsistencyLLM=Phi-4-mini (3.8B)2026.06 | 23.21 | 24.08 | |
| Temperature ScalingLLM=Gemma-3 (4B)2026.06 | 33.27 | 33.01 | |
| ConsistencyLLM=Gemma-3 (4B)2026.06 | 33.83 | 34 | |
| FSDLLM=Gemma-3 (4B)2026.06 | 33.92 | 33.92 | |
| EntropyLLM=Gemma-3 (4B)2026.06 | 33.93 | 33.92 | |
| VanillaLLM=Gemma-3 (4B)2026.06 | 34 | 34.03 |