Calibration on PopQA
0.018ECEmean ablation
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| mean ablationModel=Llama-3.2-3B-Instruct, alpha=–2026.04 | 0.018 | 0.161 | 96.9 | 67.5 | |
| steeringModel=Llama-3.2-3B-Instruct, alpha=0.62026.04 | 0.02 | 0.161 | 96.5 | 67.4 | |
| steeringModel=Llama-3.2-3B-Instruct, alpha=0.72026.04 | 0.027 | 0.163 | 95.3 | 67 | |
| steeringModel=Llama-3.2-3B-Instruct, alpha=0.82026.04 | 0.031 | 0.163 | 94.6 | 67.1 | |
| steeringModel=Llama-3.2-3B-Instruct, alpha=0.52026.04 | 0.063 | 0.146 | 88.9 | 70.5 | |
| steeringModel=Qwen2.5-3B-Instruct, alpha=0.52026.04 | 0.111 | 0.13 | 77.5 | 69.7 | |
| steeringModel=Qwen2.5-3B-Instruct, alpha=0.62026.04 | 0.145 | 0.146 | 70.7 | 66.1 | |
| steeringModel=Qwen2.5-3B-Instruct, alpha=0.42026.04 | 0.166 | 0.148 | 66.4 | 65.7 | |
| steeringModel=Qwen2.5-3B-Instruct, alpha=0.72026.04 | 0.167 | 0.166 | 66 | 61.4 | |
| steeringModel=Qwen2.5-3B-Instruct, alpha=0.82026.04 | 0.17 | 0.17 | 65.5 | 60.5 | |
| steeringModel=Llama-3.2-3B-Instruct, alpha=0.42026.04 | 0.196 | 0.189 | 65.6 | 61.9 | |
| steeringModel=Qwen2.5-3B-Instruct, alpha=0.32026.04 | 0.245 | 0.193 | 50.3 | 55.1 | |
| mean ablationModel=Qwen2.5-3B-Instruct, alpha=–2026.04 | 0.295 | 0.244 | 40 | 43.1 | |
| steeringModel=Llama-3.2-3B-Instruct, alpha=0.32026.04 | 0.338 | 0.287 | 40.7 | 42 | |
| baselineModel=Qwen2.5-3B-Instruct, alpha=–2026.04 | 0.492 | 0.43 | — | — | |
| baselineModel=Llama-3.2-3B-Instruct, alpha=–2026.04 | 0.57 | 0.495 | — | — |