Natural Language Inference on MNLI (ECE, BS)
2.74ECECaliDist (As.)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| CaliDist (As.)LLM=LLAMA-3.1 8B2026.06 | 2.74 | 19.01 | |
| CaliDist (Sa.)LLM=LLAMA-3.1 8B2026.06 | 2.77 | 19.01 | |
| VanillaLLM=LLAMA-3.1 8B2026.06 | 2.81 | — | |
| CaliDist (Pr.)LLM=LLAMA-3.1 8B2026.06 | 2.83 | 19.02 | |
| Temperature ScalingLLM=LLAMA-3.1 8B2026.06 | 5.83 | 19.27 | |
| CaliDist (Sa.)LLM=QWEN-3 8B2026.06 | 6.38 | 21.43 | |
| CaliDist (As.)LLM=QWEN-3 8B2026.06 | 6.49 | 21.36 | |
| CaliDist_verbalized (Pr.)LLM=LLAMA-3.1 8B2026.06 | 7 | 22.37 | |
| CaliDist_verbalized (Sa.)LLM=QWEN-2 7B2026.06 | 7.27 | 18.98 | |
| CaliDist_verbalized (As.)LLM=QWEN-2 7B2026.06 | 7.79 | 19.05 | |
| CaliDist_verbalized (Pr.)LLM=QWEN-2 7B2026.06 | 9.18 | 19.1 | |
| VerbalizedLLM=QWEN-2 7B2026.06 | 9.28 | 19.21 | |
| CaliDist_verbalized (As.)LLM=LLAMA-3.1 8B2026.06 | 10.03 | 23.22 | |
| CaliDist_verbalized (Sa.)LLM=LLAMA-3.1 8B2026.06 | 10.2 | 23.26 | |
| CaliDist (Pr.)LLM=QWEN-3 8B2026.06 | 11.04 | 22 | |
| ConsistencyLLM=LLAMA-3.1 8B2026.06 | 11.29 | 21.03 | |
| CaliDist_verbalized (Sa.)LLM=GEMMA-3 4B2026.06 | 11.98 | 25.72 | |
| CaliDist_verbalized (As.)LLM=GEMMA-3 4B2026.06 | 17.86 | 27.92 | |
| FSDLLM=LLAMA-3.1 8B2026.06 | 17.99 | 24.51 | |
| CaliDist_verbalized (Pr.)LLM=GEMMA-3 4B2026.06 | 19.26 | 29.31 | |
| CaliDist_verbalized (Sa.)LLM=PHI-4-MINI 3.8B2026.06 | 19.77 | 24.63 | |
| CaliDist_verbalized (Pr.)LLM=PHI-4-MINI 3.8B2026.06 | 19.91 | 27.08 | |
| CaliDist_verbalized (As.)LLM=PHI-4-MINI 3.8B2026.06 | 22.44 | 26.09 | |
| VerbalizedLLM=PHI-4-MINI 3.8B2026.06 | 24.73 | 24.98 | |
| VerbalizedLLM=LLAMA-3.1 8B2026.06 | 26.4 | 28.49 | |
| VerbalizedLLM=GEMMA-3 4B2026.06 | 26.5 | 30.42 | |
| Temperature ScalingLLM=QWEN-3 8B2026.06 | 27.16 | 28.35 | |
| VanillaLLM=QWEN-3 8B2026.06 | 29.37 | 29.87 | |
| ConsistencyLLM=QWEN-3 8B2026.06 | 30.25 | 30.54 | |
| FSDLLM=QWEN-3 8B2026.06 | 30.58 | 30.74 | |
| EntropyLLM=QWEN-3 8B2026.06 | 30.65 | 30.89 | |
| EntropyLLM=LLAMA-3.1 8B2026.06 | 32.14 | 32.44 |