Classification on MSciNLI
1.52Expected Calibration Error (ECE)CaliDist (Sa.)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CaliDist (Sa.)LLM=Gemma-3 (4B)2026.06 | 1.52 | 23.97 | |
| CaliDist (As.)LLM=Gemma-3 (4B)2026.06 | 2.6 | 25.27 | |
| CaliDist (Sa.)LLM=Phi-4-mini (3.8B)2026.06 | 2.89 | 22.68 | |
| CaliDist (Pr.)LLM=Phi-4-mini (3.8B)2026.06 | 3.57 | 23.1 | |
| CaliDist (As.)LLM=Phi-4-mini (3.8B)2026.06 | 5.14 | 22.9 | |
| CaliDist (Pr.)LLM=Gemma-3 (4B)2026.06 | 10.31 | 24.95 | |
| Temperature ScalingLLM=Phi-4-mini (3.8B)2026.06 | 18.05 | 26 | |
| FSDLLM=Phi-4-mini (3.8B)2026.06 | 26.07 | 32.6 | |
| EntropyLLM=Phi-4-mini (3.8B)2026.06 | 27.23 | 32.67 | |
| VanillaLLM=Phi-4-mini (3.8B)2026.06 | 30.22 | 32.77 | |
| ConsistencyLLM=Phi-4-mini (3.8B)2026.06 | 32.93 | 35.23 | |
| Temperature ScalingLLM=Gemma-3 (4B)2026.06 | 52.61 | 52.16 | |
| EntropyLLM=Gemma-3 (4B)2026.06 | 53.18 | 53.22 | |
| FSDLLM=Gemma-3 (4B)2026.06 | 53.39 | 53.45 | |
| VanillaLLM=Gemma-3 (4B)2026.06 | 53.9 | 53.66 | |
| ConsistencyLLM=Gemma-3 (4B)2026.06 | 53.99 | 53.84 |