Error detection on Held-out problems (n=200) (test)
0.953AUROCHidden State Probe
Evaluation results
| Method | Links | |
|---|---|---|
| Hidden State ProbeCost=1 fwd pass, Model=Qwen2.5-3B2026.05 | 0.953 | |
| Self-Consistency (N=5)Cost=5× gen, Model=Qwen2.5-3B2026.05 | 0.823 | |
| P(True)Cost=1 query, Model=Qwen2.5-3B2026.05 | 0.721 | |
| CCSCost=1 fwd pass, Model=Qwen2.5-3B2026.05 | 0.718 | |
| Sequence Log-probCost=free, Model=Qwen2.5-3B2026.05 | 0.676 | |
| Verbalized ConfidenceCost=1 query, Model=Qwen2.5-3B2026.05 | 0.674 |