Calibration on CSQA
1.04ECECaliDist (Pr.)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| CaliDist (Pr.)LLM Model=GPT-4O-MINI, Confidence Type=Log-Prob.2026.06 | 1.04 | 10.07 | |
| CaliDist (As.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Log-Prob.2026.06 | 2.39 | 10.89 | |
| CaliDist (Sa.)LLM Model=GPT-4O-MINI, Confidence Type=Log-Prob.2026.06 | 3.42 | 10.84 | |
| CaliDist (Sa.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Log-Prob.2026.06 | 3.44 | 11.36 | |
| CaliDist (As.)LLM Model=GPT-4O-MINI, Confidence Type=Log-Prob.2026.06 | 4.71 | 11.51 | |
| CaliDist (Pr.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Log-Prob.2026.06 | 5.19 | 10.71 | |
| CaliDistverb. (Sa.)LLM Model=GPT-4O-MINI, Confidence Type=Verbalized2026.06 | 7.55 | 13.27 | |
| CaliDistverb. (Pr.)LLM Model=GPT-4O-MINI, Confidence Type=Verbalized2026.06 | 7.62 | 13.38 | |
| VerbalizedLLM Model=GPT-4O-MINI, Confidence Type=Verbalized2026.06 | 8.5 | 13.08 | |
| CaliDistverb. (As.)LLM Model=GPT-4O-MINI, Confidence Type=Verbalized2026.06 | 8.59 | 13.03 | |
| VerbalizedLLM Model=GEMINI-2.0-FLASH, Confidence Type=Verbalized2026.06 | 10.56 | 13.01 | |
| CaliDistverb. (Pr.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Verbalized2026.06 | 10.56 | 13.01 | |
| CaliDistverb. (Sa.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Verbalized2026.06 | 10.56 | 13.01 | |
| CaliDistverb. (As.)LLM Model=GEMINI-2.0-FLASH, Confidence Type=Verbalized2026.06 | 11.5 | 12.88 | |
| FSDLLM Model=GEMINI-2.0-FLASH, Confidence Type=Consistency2026.06 | 12.18 | 12.8 | |
| EntropyLLM Model=GEMINI-2.0-FLASH, Confidence Type=Consistency2026.06 | 12.49 | 13.03 | |
| VanillaLLM Model=GEMINI-2.0-FLASH, Confidence Type=Log-Prob.2026.06 | 12.82 | 13.08 | |
| ConsistencyLLM Model=GEMINI-2.0-FLASH, Confidence Type=Consistency2026.06 | 12.97 | 13.21 | |
| VanillaLLM Model=GPT-4O-MINI, Confidence Type=Log-Prob.2026.06 | 13.87 | 14.21 | |
| FSDLLM Model=GPT-4O-MINI, Confidence Type=Consistency2026.06 | 14.95 | 15.82 | |
| EntropyLLM Model=GPT-4O-MINI, Confidence Type=Consistency2026.06 | 15.3 | 16 | |
| ConsistencyLLM Model=GPT-4O-MINI, Confidence Type=Consistency2026.06 | 15.52 | 16.07 |