Safety Evaluation on DirectRefusal
30.66Attack Success Rate (ASR)SafeGene
Evaluation results
| Method | Links | |
|---|---|---|
| SafeGeneBackbone=Qwen2.5-7B, Downstream Task Context=MNLI, Safety Judge=Beaver-dam-7B2026.06 | 30.66 | |
| SafeGeneBackbone=Qwen2.5-7B, Downstream Task Context=BoolQ, Safety Judge=Beaver-dam-7B2026.06 | 31.84 | |
| SafeGeneBackbone=Qwen2.5-7B, Downstream Task Context=AG News, Safety Judge=Beaver-dam-7B2026.06 | 32.63 | |
| SaLoRABackbone=Qwen2.5-7B, Downstream Task Context=BoolQ, Safety Judge=Beaver-dam-7B2026.06 | 32.76 | |
| SafeGeneBackbone=Qwen2.5-7B, Downstream Task Context=Average, Safety Judge=Beaver-dam-7B2026.06 | 33.09 | |
| SafeMERGEBackbone=Qwen2.5-7B, Downstream Task Context=MNLI, Safety Judge=Beaver-dam-7B2026.06 | 33.16 | |
| SaLoRABackbone=Qwen2.5-7B, Downstream Task Context=AG News, Safety Judge=Beaver-dam-7B2026.06 | 34.21 | |
| SaLoRABackbone=Qwen2.5-7B, Downstream Task Context=Average, Safety Judge=Beaver-dam-7B2026.06 | 34.47 | |
| SafeMERGEBackbone=Qwen2.5-7B, Downstream Task Context=BoolQ, Safety Judge=Beaver-dam-7B2026.06 | 34.61 | |
| SafeMERGEBackbone=Qwen2.5-7B, Downstream Task Context=Average, Safety Judge=Beaver-dam-7B2026.06 | 34.84 | |
| SafeMERGEBackbone=Qwen2.5-7B, Downstream Task Context=SST2, Safety Judge=Beaver-dam-7B2026.06 | 35 | |
| SaLoRABackbone=Qwen2.5-7B, Downstream Task Context=SST2, Safety Judge=Beaver-dam-7B2026.06 | 35.13 | |
| SaLoRABackbone=Qwen2.5-7B, Downstream Task Context=MNLI, Safety Judge=Beaver-dam-7B2026.06 | 35.79 | |
| SafeMERGEBackbone=Qwen2.5-7B, Downstream Task Context=AG News, Safety Judge=Beaver-dam-7B2026.06 | 36.58 | |
| SafeLoRABackbone=Qwen2.5-7B, Downstream Task Context=BoolQ, Safety Judge=Beaver-dam-7B2026.06 | 36.58 | |
| SafeLoRABackbone=Qwen2.5-7B, Downstream Task Context=AG News, Safety Judge=Beaver-dam-7B2026.06 | 37.24 | |
| SafeGeneBackbone=Qwen2.5-7B, Downstream Task Context=SST2, Safety Judge=Beaver-dam-7B2026.06 | 37.24 | |
| Fine-tunedBackbone=Qwen2.5-7B, Downstream Task Context=MNLI, Safety Judge=Beaver-dam-7B2026.06 | 38.03 | |
| SafeLoRABackbone=Qwen2.5-7B, Downstream Task Context=Average, Safety Judge=Beaver-dam-7B2026.06 | 38.42 | |
| SafeLoRABackbone=Qwen2.5-7B, Downstream Task Context=SST2, Safety Judge=Beaver-dam-7B2026.06 | 39.87 | |
| SafeLoRABackbone=Qwen2.5-7B, Downstream Task Context=MNLI, Safety Judge=Beaver-dam-7B2026.06 | 40 | |
| Fine-tunedBackbone=Qwen2.5-7B, Downstream Task Context=BoolQ, Safety Judge=Beaver-dam-7B2026.06 | 44.87 | |
| Fine-tunedBackbone=Qwen2.5-7B, Downstream Task Context=Average, Safety Judge=Beaver-dam-7B2026.06 | 45.33 | |
| Fine-tunedBackbone=Qwen2.5-7B, Downstream Task Context=AG News, Safety Judge=Beaver-dam-7B2026.06 | 45.39 | |
| Fine-tunedBackbone=Qwen2.5-7B, Downstream Task Context=SST2, Safety Judge=Beaver-dam-7B2026.06 | 53.03 |