Safety Evaluation on SecureBreak
4.44ASRMLP
Evaluation Results
| Method | Links | |
|---|---|---|
| MLPBase Model=Llama-3.1-70B, Condition=Cured2026.05 | 4.44 | |
| MoEBase Model=Llama-3.1-8B, Condition=Cured2026.05 | 6.25 | |
| MoEBase Model=Gemma-2-27B, Condition=Cured2026.05 | 6.62 | |
| MoEBase Model=Gemma-2-9b, Condition=Cured2026.05 | 6.94 | |
| MLPBase Model=Qwen2.5-72B, Condition=Cured2026.05 | 8.38 | |
| MoEBase Model=Mistral-7B-v0.3, Condition=Cured2026.05 | 8.56 | |
| MLPBase Model=Mistral-7B-v0.3, Condition=Cured2026.05 | 10.44 | |
| FMBase Model=Gemma-2-9b, Condition=Cured2026.05 | 10.62 | |
| MLPBase Model=Qwen2.5-7B, Condition=Cured2026.05 | 10.88 | |
| MoEBase Model=Qwen2.5-7B, Condition=Cured2026.05 | 11.31 | |
| MLPBase Model=Gemma-2-27B, Condition=Cured2026.05 | 11.5 | |
| FMBase Model=Gemma-2-27B, Condition=Cured2026.05 | 12.62 | |
| FMBase Model=Llama-3.1-8B, Condition=Cured2026.05 | 17 | |
| MLPBase Model=Llama-3.1-8B, Condition=Cured2026.05 | 18.5 | |
| MoEBase Model=Qwen2.5-72B, Condition=Cured2026.05 | 20.31 | |
| MLPBase Model=Gemma-2-9b, Condition=Cured2026.05 | 20.38 | |
| MoEBase Model=Llama-3.1-70B, Condition=Cured2026.05 | 20.44 | |
| FMBase Model=Mistral-7B-v0.3, Condition=Cured2026.05 | 25.69 | |
| FMBase Model=Qwen2.5-7B, Condition=Cured2026.05 | 26.06 | |
| CVAEBase Model=Qwen2.5-72B, Condition=Cured2026.05 | 31 | |
| CVAEBase Model=Gemma-2-9b, Condition=Cured2026.05 | 31.37 | |
| FMBase Model=Qwen2.5-72B, Condition=Cured2026.05 | 32.12 | |
| AEBase Model=Gemma-2-27B, Condition=Cured2026.05 | 32.31 | |
| AEBase Model=Gemma-2-9b, Condition=Cured2026.05 | 37.69 | |
| FMBase Model=Llama-3.1-70B, Condition=Cured2026.05 | 38 | |
| AEBase Model=Qwen2.5-72B, Condition=Cured2026.05 | 40.25 | |
| Safe LoRABase Model=Mistral-7B-v0.3, Condition=Cured2026.05 | 40.75 | |
| CVAEBase Model=Mistral-7B-v0.3, Condition=Cured2026.05 | 40.75 | |
| Safe LoRABase Model=Gemma-2-9b, Condition=Cured2026.05 | 41.38 | |
| AEBase Model=Qwen2.5-7B, Condition=Cured2026.05 | 41.81 | |
| AEBase Model=Mistral-7B-v0.3, Condition=Cured2026.05 | 43.38 | |
| CVAEBase Model=Gemma-2-27B, Condition=Cured2026.05 | 43.62 | |
| SaLoRABase Model=Qwen2.5-72B, Condition=Cured2026.05 | 49.75 | |
| Safe LoRABase Model=Llama-3.1-8B, Condition=Cured2026.05 | 49.94 | |
| Unsafe BaselineBase Model=Qwen2.5-72B, Condition=Unsafe (Finetuned)2026.05 | 50 | |
| CVAEBase Model=Llama-3.1-70B, Condition=Cured2026.05 | 50.25 | |
| CVAEBase Model=Llama-3.1-8B, Condition=Cured2026.05 | 51 | |
| AEBase Model=Llama-3.1-70B, Condition=Cured2026.05 | 51.06 | |
| AEBase Model=Llama-3.1-8B, Condition=Cured2026.05 | 51.5 | |
| Unsafe BaselineBase Model=Gemma-2-9b, Condition=Unsafe (Finetuned)2026.05 | 51.88 | |
| SaLoRABase Model=Gemma-2-9b, Condition=Cured2026.05 | 51.94 | |
| Safe LoRABase Model=Qwen2.5-72B, Condition=Cured2026.05 | 53.37 | |
| Unsafe BaselineBase Model=Gemma-2-27B, Condition=Unsafe (Finetuned)2026.05 | 53.44 | |
| SaLoRABase Model=Mistral-7B-v0.3, Condition=Cured2026.05 | 53.5 | |
| Unsafe BaselineBase Model=Mistral-7B-v0.3, Condition=Unsafe (Finetuned)2026.05 | 53.87 | |
| SaLoRABase Model=Gemma-2-27B, Condition=Cured2026.05 | 54.25 | |
| SaLoRABase Model=Llama-3.1-70B, Condition=Cured2026.05 | 58.19 | |
| Unsafe BaselineBase Model=Qwen2.5-7B, Condition=Unsafe (Finetuned)2026.05 | 58.81 | |
| Unsafe BaselineBase Model=Llama-3.1-70B, Condition=Unsafe (Finetuned)2026.05 | 58.81 | |
| SaLoRABase Model=Qwen2.5-7B, Condition=Cured2026.05 | 58.88 | |
| Safe LoRABase Model=Llama-3.1-70B, Condition=Cured2026.05 | 59.56 | |
| Unsafe BaselineBase Model=Llama-3.1-8B, Condition=Unsafe (Finetuned)2026.05 | 60.56 | |
| SaLoRABase Model=Llama-3.1-8B, Condition=Cured2026.05 | 62 | |
| Safe LoRABase Model=Gemma-2-27B, Condition=Cured2026.05 | 62.81 | |
| CVAEBase Model=Qwen2.5-7B, Condition=Cured2026.05 | 64.44 | |
| Safe LoRABase Model=Qwen2.5-7B, Condition=Cured2026.05 | 72 |