Attack Success Evaluation on Beavertails
56.7Attack Success RateVaccine
Evaluation results
| Method | Links | |
|---|---|---|
| VaccineBase Model=Llama3-8B2026.06 | 56.7 | |
| BoosterBase Model=Llama3-8B2026.06 | 49.1 | |
| VaccineBase Model=Qwen2.5-1.5B2026.06 | 48.5 | |
| VaccineTest-time Attack Dataset=Beavertails-unsafe2026.06 | 48.5 | |
| SFTBase Model=Llama3-8B2026.06 | 48.3 | |
| BoosterBase Model=Qwen3-4B2026.06 | 44.9 | |
| VaccineTest-time Attack Dataset=PKU-SafeRLHF-unsafe2026.06 | 44.4 | |
| VaccineBase Model=Qwen3-4B2026.06 | 44 | |
| BoosterBase Model=Qwen2.5-1.5B2026.06 | 43.4 | |
| BoosterTest-time Attack Dataset=Beavertails-unsafe2026.06 | 43.4 | |
| SFTBase Model=Qwen3-4B2026.06 | 39.7 | |
| SFTBase Model=Qwen2.5-1.5B2026.06 | 38.8 | |
| SFTTest-time Attack Dataset=Beavertails-unsafe2026.06 | 38.8 | |
| BoosterTest-time Attack Dataset=PKU-SafeRLHF-unsafe2026.06 | 37.8 | |
| VaccineTest-time Attack Dataset=Toxic-DPO2026.06 | 36.1 | |
| SFTTest-time Attack Dataset=PKU-SafeRLHF-unsafe2026.06 | 29.8 | |
| BoosterTest-time Attack Dataset=Toxic-DPO2026.06 | 29.8 | |
| PatcherBase Model=Llama3-8B2026.06 | 26 | |
| SFTTest-time Attack Dataset=Toxic-DPO2026.06 | 25.2 | |
| PatcherBase Model=Qwen3-4B2026.06 | 20.4 | |
| PatcherTest-time Attack Dataset=PKU-SafeRLHF-unsafe2026.06 | 18.8 | |
| PatcherBase Model=Qwen2.5-1.5B2026.06 | 18.1 | |
| PatcherTest-time Attack Dataset=Beavertails-unsafe2026.06 | 18.1 | |
| PatcherTest-time Attack Dataset=Toxic-DPO2026.06 | 15.7 |