Multiple-choice question answering on 2,000 questions dataset
0.029ECEP(True) Norm
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| P(True) NormModel=Llama 3.2-3B, Temperature Scaling=true2026.03 | 0.029 | 0.03 | 0.218 | 0.692 | 57.6 | |
| P(True) NormModel=Gemma 2-2B, Temperature Scaling=true2026.03 | 0.037 | 0.033 | 0.233 | 0.647 | 51.6 | |
| VerbalizedModel=Llama 3.2-3B, Temperature Scaling=true2026.03 | 0.047 | 0.075 | 0.25 | 0.504 | 57.6 | |
| VerbalizedModel=Gemma 2-2B, Temperature Scaling=true2026.03 | 0.047 | 0.037 | 0.249 | 0.55 | 51.6 | |
| VerbalizedModel=Phi 3.5-Mini (3.8B), Temperature Scaling=true2026.03 | 0.047 | 0.049 | 0.215 | 0.583 | 66.7 | |
| SECLModel=Llama 3.2-3B2026.03 | 0.05 | 0.06 | 0.241 | 0.587 | 57.7 | |
| SECLModel=Gemma 2-2B2026.03 | 0.056 | 0.06 | 0.254 | 0.548 | 51.5 | |
| P(True) NormModel=Phi 3.5-Mini (3.8B), Temperature Scaling=true2026.03 | 0.059 | 0.06 | 0.205 | 0.664 | 66.7 | |
| P(True) NormModel=Llama 3.2-3B, Temperature Scaling=false2026.03 | 0.065 | 0.067 | 0.223 | 0.694 | 57.6 | |
| VerbalizedModel=Llama 3.1-8B, Temperature Scaling=true2026.03 | 0.08 | 0.089 | 0.213 | 0.681 | 64.4 | |
| SECLModel=Llama 3.1-8B2026.03 | 0.083 | 0.08 | 0.222 | 0.643 | 64.6 | |
| Self-ConsistencyModel=Llama 3.2-3B, Sampling passes=102026.03 | 0.093 | 0.096 | 0.211 | 0.728 | 62.4 | |
| P(True) NormModel=Llama 3.1-8B, Temperature Scaling=true2026.03 | 0.108 | 0.103 | 0.208 | 0.716 | 64.6 | |
| SECLModel=Phi 3.5-Mini (3.8B)2026.03 | 0.11 | 0.119 | 0.251 | 0.521 | 66.5 | |
| P(True) NormModel=Llama 3.1-8B, Temperature Scaling=false2026.03 | 0.12 | 0.117 | 0.211 | 0.718 | 64.6 | |
| P(True) NormModel=Gemma 2-2B, Temperature Scaling=false2026.03 | 0.141 | 0.142 | 0.259 | 0.65 | 51.6 | |
| P(True) NormModel=Phi 3.5-Mini (3.8B), Temperature Scaling=false2026.03 | 0.154 | 0.151 | 0.227 | 0.675 | 66.7 | |
| VerbalizedModel=Llama 3.2-3B, Temperature Scaling=false2026.03 | 0.17 | 0.167 | 0.292 | 0.51 | 57.6 | |
| VerbalizedModel=Llama 3.1-8B, Temperature Scaling=false2026.03 | 0.225 | 0.225 | 0.258 | 0.684 | 64.4 | |
| VerbalizedModel=Phi 3.5-Mini (3.8B), Temperature Scaling=false2026.03 | 0.251 | 0.24 | 0.275 | 0.6 | 66.7 | |
| VerbalizedModel=Gemma 2-2B, Temperature Scaling=false2026.03 | 0.256 | 0.252 | 0.314 | 0.558 | 51.6 |