Safety Evaluation on Harmbench (ASR)
3.5ASRBase Model
Evaluation Results
| Method | Links | |
|---|---|---|
| Base ModelBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=None, Filtering Method=None2026.05 | 3.5 | |
| DataShieldBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=Ours2026.05 | 5.5 | |
| Base ModelBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=None, Filtering Method=None2026.05 | 6.5 | |
| LARFBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=LARF2026.05 | 7.5 | |
| Base ModelBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=None, Filtering Method=None2026.05 | 9.5 | |
| SEALBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=SEAL2026.05 | 9.5 | |
| Bi-AnchoringBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=Bi-Anchoring2026.05 | 10.5 | |
| DataShieldBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=Ours2026.05 | 12.5 | |
| DataShieldBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=Ours2026.05 | 14.5 | |
| GradSafeBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=GradSafe2026.05 | 17.5 | |
| LARFBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=LARF2026.05 | 18 | |
| LARFBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=LARF2026.05 | 18.5 | |
| SEALBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=SEAL2026.05 | 21.5 | |
| Without FilteringBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=Without Filtering2026.05 | 21.5 | |
| SEALBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=SEAL2026.05 | 28 | |
| Bi-AnchoringBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=Bi-Anchoring2026.05 | 34 | |
| DataShieldBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=Ours2026.05 | 37.5 | |
| GradSafeBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=GradSafe2026.05 | 42 | |
| GradSafeBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=GradSafe2026.05 | 44.4 | |
| Without FilteringBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=Without Filtering2026.05 | 47.5 | |
| DataShieldBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=Ours2026.05 | 48 | |
| Without FilteringBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=Without Filtering2026.05 | 51.5 | |
| LARFBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=LARF2026.05 | 52 | |
| LARFBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=LARF2026.05 | 54 | |
| Bi-AnchoringBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=Alpaca Dataset, Filtering Method=Bi-Anchoring2026.05 | 54.5 | |
| SEALBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=SEAL2026.05 | 55 | |
| SEALBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=SEAL2026.05 | 56.2 | |
| DataShieldBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=Ours2026.05 | 56.5 | |
| LARFBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=LARF2026.05 | 61 | |
| SEALBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=SEAL2026.05 | 65.5 | |
| Bi-AnchoringBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=Bi-Anchoring2026.05 | 66 | |
| Bi-AnchoringBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=Bi-Anchoring2026.05 | 70.5 | |
| GradSafeBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=GradSafe2026.05 | 76 | |
| Without FilteringBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=Without Filtering2026.05 | 77.5 | |
| Standard SFTBackbone=Llama3.1-8B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=Standard SFT2026.05 | 80 | |
| GradSafeBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=GradSafe2026.05 | 81.5 | |
| GradSafeBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=GradSafe2026.05 | 81.5 | |
| Bi-AnchoringBackbone=Llama3-8B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=Bi-Anchoring2026.05 | 83.5 | |
| Without FilteringBackbone=Qwen2.5-7B-Instruct, Fine-tuning Dataset=Dolly Dataset, Filtering Method=Without Filtering2026.05 | 83.5 |