Expected Calibration Error on ARC-C
5ECE (15-bin)PPT (Distribution)
Evaluation results
| Method | Links | |
|---|---|---|
| PPT (Distribution)Model=QWEN2-7B, Temperature Scaling=true2026.05 | 5 | |
| PPT (Distribution)Model=LLAMA3-8B, Temperature Scaling=true2026.05 | 5.1 | |
| PPT (Distribution)Model=LLAMA3-8B, Temperature Scaling=false2026.05 | 5.8 | |
| BaseModel=LLAMA3-8B, Temperature Scaling=true2026.05 | 6.2 | |
| BaseModel=QWEN2-7B, Temperature Scaling=true2026.05 | 6.2 | |
| PPT (Distribution)Model=QWEN2-7B, Temperature Scaling=false2026.05 | 9.2 | |
| PPT (Mean)Model=LLAMA3-8B, Temperature Scaling=false2026.05 | 11.8 | |
| PPT (Mean)Model=QWEN2-7B, Temperature Scaling=false2026.05 | 12.8 | |
| BaseModel=QWEN2-7B, Temperature Scaling=false2026.05 | 13.5 | |
| BaseModel=LLAMA3-8B, Temperature Scaling=false2026.05 | 15.8 | |
| VerbalizedModel=LLAMA3-8B, Temperature Scaling=false2026.05 | 21.1 |