Expected Calibration Error on BT
0.025ECE (15-bin)PPT (Distribution)
Evaluation Results
| Method | Links | |
|---|---|---|
| PPT (Distribution)Model=LLAMA3-8B, Temperature Scaling=true2026.05 | 0.025 | |
| PPT (Distribution)Model=QWEN2-7B, Temperature Scaling=true2026.05 | 0.056 | |
| PPT (Distribution)Model=LLAMA3-8B, Temperature Scaling=false2026.05 | 0.218 | |
| VerbalizedModel=LLAMA3-8B, Temperature Scaling=false2026.05 | 0.255 | |
| PPT (Distribution)Model=QWEN2-7B, Temperature Scaling=false2026.05 | 0.278 | |
| PPT (Mean)Model=LLAMA3-8B, Temperature Scaling=false2026.05 | 0.38 | |
| PPT (Mean)Model=QWEN2-7B, Temperature Scaling=false2026.05 | 0.415 | |
| BaseModel=QWEN2-7B, Temperature Scaling=false2026.05 | 0.464 | |
| BaseModel=LLAMA3-8B, Temperature Scaling=false2026.05 | 0.467 |