LLM Safety on Safety Evaluation Set
1.66Harmful Response RateProbing Vector
Evaluation Results
| Method | Links | |
|---|---|---|
| Probing VectorIntervention Type=Switching, Intervention Size=300002026.02 | 1.66 | |
| Gradient (LESS)Intervention Type=Switching, Intervention Size=300002026.02 | 2.51 | |
| Gradient (LESS)Intervention Type=Switching, Intervention Size=120002026.02 | 2.57 | |
| Probing VectorIntervention Type=Filtering, Intervention Size=300002026.02 | 2.86 | |
| Gradient (LESS)Intervention Type=Filtering, Intervention Size=120002026.02 | 3.1 | |
| Probing VectorIntervention Type=Filtering, Intervention Size=120002026.02 | 3.3 | |
| LLM ToxicIntervention Type=Filtering, Intervention Size=30002026.02 | 3.48 | |
| LLM ToxicIntervention Type=Filtering, Intervention Size=300002026.02 | 3.61 | |
| LLM ToxicIntervention Type=Switching, Intervention Size=300002026.02 | 3.73 | |
| Gradient (LESS)Intervention Type=Filtering, Intervention Size=30002026.02 | 3.75 | |
| LLM ToxicIntervention Type=Filtering, Intervention Size=120002026.02 | 3.9 | |
| LLM ToxicIntervention Type=Switching, Intervention Size=120002026.02 | 4.63 | |
| RandomIntervention Type=Switching, Intervention Size=300002026.02 | 4.96 | |
| Probing VectorIntervention Type=Filtering, Intervention Size=30002026.02 | 5.13 | |
| RandomIntervention Type=Switching, Intervention Size=120002026.02 | 5.16 | |
| Gradient (LESS)Intervention Type=Filtering, Intervention Size=300002026.02 | 5.78 | |
| Probing VectorIntervention Type=Switching, Intervention Size=120002026.02 | 6.33 | |
| Probing VectorIntervention Type=Switching, Intervention Size=30002026.02 | 6.38 | |
| RandomIntervention Type=Filtering, Intervention Size=300002026.02 | 6.67 | |
| Gradient (LESS)Intervention Type=Switching, Intervention Size=30002026.02 | 6.71 | |
| RandomIntervention Type=Filtering, Intervention Size=30002026.02 | 6.78 | |
| LLM ToxicIntervention Type=Switching, Intervention Size=30002026.02 | 6.89 | |
| Baseline (DPO, no intervention)Intervention Type=None, Intervention Size=02026.02 | 7.63 | |
| RandomIntervention Type=Switching, Intervention Size=30002026.02 | 7.68 | |
| RandomIntervention Type=Filtering, Intervention Size=120002026.02 | 8.18 |