List your company
ResearchFormerly WizWand

Calibration on HellaSwag

3.18Expected Calibration Error (ECE)CaliDist (As.)
2.40327.646612.8918.1334Jun 4, 2026
Updated 4mo ago

Evaluation results

MethodLinks
CaliDist (As.)LLM Model=GPT-4O-MINI, Confidence Type=Log-Prob.2026.063.1810.84
CaliDist (Pr.)LLM Model=GPT-4O-MINI, Confidence Type=Log-Prob.2026.063.3610.48
CaliDist (Pr.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Log-Prob.2026.064.98.27
CaliDist (Sa.)LLM Model=GPT-4O-MINI, Confidence Type=Log-Prob.2026.065.0711.04
CaliDist (As.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Log-Prob.2026.065.168.55
CaliDist (Sa.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Log-Prob.2026.065.898.69
EntropyLLM Model=GEMINI-2.0-FLASH, Confidence Type=Consistency2026.068.769.16
FSDLLM Model=GEMINI-2.0-FLASH, Confidence Type=Consistency2026.068.799.11
ConsistencyLLM Model=GEMINI-2.0-FLASH, Confidence Type=Consistency2026.069.239.39
FSDLLM Model=GPT-4O-MINI, Confidence Type=Consistency2026.069.3711.51
VanillaLLM Model=GEMINI-2.0-FLASH, Confidence Type=Log-Prob.2026.069.889.61
ConsistencyLLM Model=GPT-4O-MINI, Confidence Type=Consistency2026.0610.911.99
EntropyLLM Model=GPT-4O-MINI, Confidence Type=Consistency2026.0611.0412.31
VanillaLLM Model=GPT-4O-MINI, Confidence Type=Log-Prob.2026.0612.9613.14
VerbalizedLLM Model=GPT-4O-MINI, Confidence Type=Verbalized2026.0613.7914.61
CaliDistverb. (Sa.)LLM Model=GPT-4O-MINI, Confidence Type=Verbalized2026.0613.7914.59
CaliDistverb. (As.)LLM Model=GPT-4O-MINI, Confidence Type=Verbalized2026.0614.3814.61
CaliDistverb. (Pr.)LLM Model=GPT-4O-MINI, Confidence Type=Verbalized2026.0614.8314.26
VerbalizedLLM Model=GEMINI-2.0-FLASH, Confidence Type=Verbalized2026.0622.613.8
CaliDistverb. (As.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Verbalized2026.0622.613.8
CaliDistverb. (Pr.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Verbalized2026.0622.613.8
CaliDistverb. (Sa.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Verbalized2026.0622.613.8