Refusal Detection on Alpaca + XSTest (val)
0.988AUROCwopt
Evaluation results
| Method | Links | ||
|---|---|---|---|
| woptModel=Qwen2.5 (instruct), Layer=222026.04 | 0.988 | 83.4 | |
| theta two-classModel=Qwen3.5 (base), Layer=102026.04 | 0.988 | 81.5 | |
| woptModel=Qwen2.5 (base), Layer=222026.04 | 0.987 | 83.6 | |
| wLDAModel=Qwen3.5 (base), Layer=102026.04 | 0.987 | 83.3 | |
| woptModel=Qwen3.5 (base), Layer=102026.04 | 0.987 | 84 | |
| woptModel=Qwen2.5 (abliterated), Layer=222026.04 | 0.981 | 71.8 | |
| woptModel=Qwen3.5 (instruct), Layer=222026.04 | 0.979 | 79.4 | |
| wLDAModel=Qwen2.5 (instruct), Layer=222026.04 | 0.975 | 65.4 | |
| wLDAModel=Qwen2.5 (base), Layer=222026.04 | 0.974 | 65.7 | |
| woptModel=Qwen3.5 (abliterated), Layer=222026.04 | 0.974 | 75.4 | |
| theta two-classModel=Qwen2.5 (base), Layer=222026.04 | 0.966 | 52.5 | |
| theta two-classModel=Qwen2.5 (instruct), Layer=222026.04 | 0.966 | 54.2 | |
| wLDAModel=Qwen3.5 (instruct), Layer=222026.04 | 0.964 | 64.2 | |
| wLDAModel=Qwen2.5 (abliterated), Layer=222026.04 | 0.961 | 53 | |
| wLDAModel=Qwen3.5 (abliterated), Layer=222026.04 | 0.958 | 57.5 | |
| theta two-classModel=Qwen2.5 (abliterated), Layer=222026.04 | 0.945 | 36 | |
| theta two-classModel=Qwen3.5 (instruct), Layer=222026.04 | 0.915 | 45.7 | |
| theta two-classModel=Qwen3.5 (abliterated), Layer=222026.04 | 0.902 | 42.2 | |
| theta-normModel=Qwen3.5 (abliterated), Layer=222026.04 | 0.77 | 0.1 | |
| wPC1Model=Qwen3.5 (base), Layer=102026.04 | 0.76 | 2.2 | |
| theta-normModel=Qwen3.5 (instruct), Layer=222026.04 | 0.758 | 0.3 | |
| RandomModel=Qwen3.5 (instruct), Layer=222026.04 | 0.694 | 2.7 | |
| RandomModel=Qwen3.5 (abliterated), Layer=222026.04 | 0.691 | 2.7 | |
| theta-normModel=Qwen2.5 (abliterated), Layer=222026.04 | 0.679 | 0.3 | |
| RandomModel=Qwen3.5 (base), Layer=102026.04 | 0.675 | 3.3 | |
| PerplexityModel=Qwen2.5 (abliterated), Layer=—2026.04 | 0.667 | 0.1 | |
| theta-normModel=Qwen2.5 (instruct), Layer=222026.04 | 0.661 | 0.3 | |
| PerplexityModel=Qwen3.5 (abliterated), Layer=—2026.04 | 0.618 | 0.3 | |
| PerplexityModel=Qwen2.5 (base), Layer=—2026.04 | 0.611 | 0.3 | |
| PerplexityModel=Qwen2.5 (instruct), Layer=—2026.04 | 0.609 | 0 | |
| wPC1Model=Qwen2.5 (base), Layer=222026.04 | 0.607 | 0.3 | |
| theta-normModel=Qwen2.5 (base), Layer=222026.04 | 0.605 | 0 | |
| RandomModel=Qwen2.5 (instruct), Layer=222026.04 | 0.604 | 2.2 | |
| theta-normModel=Qwen3.5 (base), Layer=102026.04 | 0.602 | 0 | |
| wPC1Model=Qwen2.5 (abliterated), Layer=222026.04 | 0.6 | 0.3 | |
| RandomModel=Qwen2.5 (base), Layer=222026.04 | 0.589 | 1 | |
| PerplexityModel=Qwen3.5 (base), Layer=—2026.04 | 0.589 | 0.1 | |
| PerplexityModel=Qwen3.5 (instruct), Layer=—2026.04 | 0.584 | 0.1 | |
| wPC1Model=Qwen2.5 (instruct), Layer=222026.04 | 0.578 | 0.1 | |
| wPC1Model=Qwen3.5 (abliterated), Layer=222026.04 | 0.568 | 0 | |
| wPC1Model=Qwen3.5 (instruct), Layer=222026.04 | 0.567 | 0.1 | |
| RandomModel=Qwen2.5 (abliterated), Layer=222026.04 | 0.542 | 2.1 |