Confidence Calibration on Average of four domains Relational Inference Planning
0.114Brier Scorefirst-second-distance-based (FSD)
Evaluation results
| Method | Links | |
|---|---|---|
| first-second-distance-based (FSD)Language Model=GPT-4, Prompting Strategy=Averaged across 5 strategies, Number of samples (n)=40, Temperature (T)=0.42024.02 | 0.114 | |
| entropy-based consistencyLanguage Model=GPT-4, Prompting Strategy=Averaged across 5 strategies, Number of samples (n)=40, Temperature (T)=0.42024.02 | 0.116 | |
| agreement-based consistencyLanguage Model=GPT-4, Prompting Strategy=Averaged across 5 strategies, Number of samples (n)=40, Temperature (T)=0.42024.02 | 0.119 | |
| agreement-based consistencyLanguage Model=Codex, Prompting Strategy=Averaged across 5 strategies, Number of samples (n)=40, Temperature (T)=0.42024.02 | 0.151 | |
| verb_lingLanguage Model=GPT-4, Prompting Strategy=Averaged across 5 strategies2024.02 | 0.154 | |
| first-second-distance-based (FSD)Language Model=Codex, Prompting Strategy=Averaged across 5 strategies, Number of samples (n)=40, Temperature (T)=0.42024.02 | 0.159 | |
| entropy-based consistencyLanguage Model=Codex, Prompting Strategy=Averaged across 5 strategies, Number of samples (n)=40, Temperature (T)=0.42024.02 | 0.175 | |
| p_true (8-shot)Language Model=Codex, Prompting Strategy=Averaged across 5 strategies2024.02 | 0.179 | |
| verb_percentLanguage Model=GPT-4, Prompting Strategy=Averaged across 5 strategies2024.02 | 0.181 | |
| p_true (0-shot)Language Model=Codex, Prompting Strategy=Averaged across 5 strategies2024.02 | 0.188 | |
| entropy-based consistencyLanguage Model=GPT-3.5-turbo, Prompting Strategy=Averaged across 5 strategies, Number of samples (n)=40, Temperature (T)=0.42024.02 | 0.205 | |
| first-second-distance-based (FSD)Language Model=GPT-3.5-turbo, Prompting Strategy=Averaged across 5 strategies, Number of samples (n)=40, Temperature (T)=0.42024.02 | 0.207 | |
| logitLanguage Model=Codex, Prompting Strategy=Averaged across 5 strategies2024.02 | 0.209 | |
| agreement-based consistencyLanguage Model=GPT-3.5-turbo, Prompting Strategy=Averaged across 5 strategies, Number of samples (n)=40, Temperature (T)=0.42024.02 | 0.221 | |
| verb_lingLanguage Model=Codex, Prompting Strategy=Averaged across 5 strategies2024.02 | 0.249 | |
| verb_percentLanguage Model=Codex, Prompting Strategy=Averaged across 5 strategies2024.02 | 0.249 | |
| verb_lingLanguage Model=GPT-3.5-turbo, Prompting Strategy=Averaged across 5 strategies2024.02 | 0.271 | |
| verb_percentLanguage Model=GPT-3.5-turbo, Prompting Strategy=Averaged across 5 strategies2024.02 | 0.273 |