List your company
ResearchFormerly WizWand

Calibration on MSciNLI

3.75Expected Calibration Error (ECE)CaliDistverb. (Sa.)
2.501210.930619.3627.7894Jun 4, 2026
Updated 4mo ago

Evaluation results

MethodLinks
CaliDistverb. (Sa.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Verbalized2026.063.7520.79
CaliDist (Sa.)LLM Model=GPT-4O-MINI, Confidence Type=Log-Prob.2026.063.9222.58
CaliDist (As.)LLM Model=GPT-4O-MINI, Confidence Type=Log-Prob.2026.064.0121.83
CaliDist (Pr.)LLM Model=GPT-4O-MINI, Confidence Type=Log-Prob.2026.064.0621.59
CaliDist (Pr.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Log-Prob.2026.064.8218.88
CaliDistverb. (As.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Verbalized2026.064.8921.09
VerbalizedLLM Model=GEMINI-2.0-FLASH, Confidence Type=Verbalized2026.064.9121.66
CaliDistverb. (Pr.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Verbalized2026.065.0421.27
CaliDist (As.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Log-Prob.2026.065.4620.44
CaliDist (Sa.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Log-Prob.2026.065.8721.03
CaliDistverb. (Sa.)LLM Model=GPT-4O-MINI, Confidence Type=Verbalized2026.0611.4724.89
VerbalizedLLM Model=GPT-4O-MINI, Confidence Type=Verbalized2026.0613.2524.98
CaliDistverb. (As.)LLM Model=GPT-4O-MINI, Confidence Type=Verbalized2026.0613.5924.53
CaliDistverb. (Pr.)LLM Model=GPT-4O-MINI, Confidence Type=Verbalized2026.0613.9625.73
EntropyLLM Model=GPT-4O-MINI, Confidence Type=Consistency2026.0625.2329.36
FSDLLM Model=GEMINI-2.0-FLASH, Confidence Type=Consistency2026.0628.0929.59
EntropyLLM Model=GEMINI-2.0-FLASH, Confidence Type=Consistency2026.0628.1429.69
FSDLLM Model=GPT-4O-MINI, Confidence Type=Consistency2026.0628.7630.77
VanillaLLM Model=GEMINI-2.0-FLASH, Confidence Type=Log-Prob.2026.0629.8130.38
ConsistencyLLM Model=GEMINI-2.0-FLASH, Confidence Type=Consistency2026.0630.2830.75
ConsistencyLLM Model=GPT-4O-MINI, Confidence Type=Consistency2026.0633.233.28
VanillaLLM Model=GPT-4O-MINI, Confidence Type=Log-Prob.2026.0634.9735.14