Expected Calibration Error on HellaSwag
1.5ECE (15-bin)PPT (Distribution)
Evaluation results
| Method | Links | |
|---|---|---|
| PPT (Distribution)Model=QWEN2-7B, Temperature Scaling=true2026.05 | 1.5 | |
| BaseModel=LLAMA3-8B, Temperature Scaling=true2026.05 | 2.3 | |
| PPT (Distribution)Model=LLAMA3-8B, Temperature Scaling=true2026.05 | 3.2 | |
| BaseModel=QWEN2-7B, Temperature Scaling=true2026.05 | 3.5 | |
| PPT (Mean)Model=LLAMA3-8B, Temperature Scaling=false2026.05 | 6 | |
| PPT (Distribution)Model=QWEN2-7B, Temperature Scaling=false2026.05 | 6 | |
| PPT (Distribution)Model=LLAMA3-8B, Temperature Scaling=false2026.05 | 6.4 | |
| PPT (Mean)Model=QWEN2-7B, Temperature Scaling=false2026.05 | 11.9 | |
| BaseModel=QWEN2-7B, Temperature Scaling=false2026.05 | 13.4 | |
| VerbalizedModel=LLAMA3-8B, Temperature Scaling=false2026.05 | 14.1 | |
| BaseModel=LLAMA3-8B, Temperature Scaling=false2026.05 | 19.6 |