Expected Calibration Error on Winogrande
8ECE (15-bin)PPT (Distribution)
Evaluation results
| Method | Links | |
|---|---|---|
| PPT (Distribution)Model=LLAMA3-8B, Temperature Scaling=true2026.05 | 8 | |
| PPT (Distribution)Model=LLAMA3-8B, Temperature Scaling=false2026.05 | 9.4 | |
| PPT (Distribution)Model=QWEN2-7B, Temperature Scaling=true2026.05 | 11 | |
| BaseModel=LLAMA3-8B, Temperature Scaling=true2026.05 | 12.6 | |
| BaseModel=QWEN2-7B, Temperature Scaling=true2026.05 | 12.8 | |
| PPT (Mean)Model=LLAMA3-8B, Temperature Scaling=false2026.05 | 22.1 | |
| PPT (Distribution)Model=QWEN2-7B, Temperature Scaling=false2026.05 | 24.6 | |
| VerbalizedModel=LLAMA3-8B, Temperature Scaling=false2026.05 | 28.4 | |
| BaseModel=LLAMA3-8B, Temperature Scaling=false2026.05 | 28.9 | |
| PPT (Mean)Model=QWEN2-7B, Temperature Scaling=false2026.05 | 32.4 | |
| BaseModel=QWEN2-7B, Temperature Scaling=false2026.05 | 33.7 |