Safety Evaluation on BeaverTails (ASR, HS)
8.8ASRSPARD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SPARDModel=Qwen-3-8B2026.05 | 8.8 | 1.28 | |
| SPARDModel=Qwen-2.5-14B-Instruct2026.05 | 11.6 | 1.38 | |
| SafeGradModel=Qwen-3-8B2026.05 | 16.6 | 1.45 | |
| SPARDTraining=Full fine-tuning, Backbone=SmolLM2-1.7B-Instruct2026.05 | 17.4 | 1.55 | |
| LisaModel=Qwen-3-8B2026.05 | 19.6 | 1.53 | |
| LisaModel=Qwen-2.5-14B-Instruct2026.05 | 19.8 | 1.55 | |
| SafeGradModel=Qwen-2.5-14B-Instruct2026.05 | 20.8 | 1.6 | |
| SPARDBackbone=Qwen-2.5-7B-Instruct2026.05 | 21.8 | 1.65 | |
| SafeGradTraining=Full fine-tuning, Backbone=SmolLM2-1.7B-Instruct2026.05 | 23.2 | 1.71 | |
| SFTModel=Qwen-2.5-14B-Instruct2026.05 | 24 | 1.68 | |
| Qwen-2.5-7B-InstructBackbone=Qwen-2.5-7B-Instruct2026.05 | 25.8 | 1.71 | |
| LisaBackbone=Qwen-2.5-7B-Instruct2026.05 | 26 | 1.74 | |
| SafeGradBackbone=Qwen-2.5-7B-Instruct2026.05 | 28.8 | 1.8 | |
| SafeGeneBackbone=Qwen2.5-7B, Downstream Task Context=BoolQ, Safety Judge=Beaver-dam-7B2026.06 | 34.16 | — | |
| SafeGeneBackbone=Qwen2.5-7B, Downstream Task Context=AG News, Safety Judge=Beaver-dam-7B2026.06 | 34.28 | — | |
| SafeGeneBackbone=Qwen2.5-7B, Downstream Task Context=Average, Safety Judge=Beaver-dam-7B2026.06 | 34.58 | — | |
| SafeGeneBackbone=Qwen2.5-7B, Downstream Task Context=SST2, Safety Judge=Beaver-dam-7B2026.06 | 34.85 | — | |
| SafeGeneBackbone=Qwen2.5-7B, Downstream Task Context=MNLI, Safety Judge=Beaver-dam-7B2026.06 | 35.03 | — | |
| PTSTBackbone=Qwen-2.5-7B-Instruct2026.05 | 37.2 | 2.09 | |
| SaLoRABackbone=Qwen2.5-7B, Downstream Task Context=BoolQ, Safety Judge=Beaver-dam-7B2026.06 | 38.32 | — | |
| SaLoRABackbone=Qwen2.5-7B, Downstream Task Context=SST2, Safety Judge=Beaver-dam-7B2026.06 | 38.78 | — | |
| SaLoRABackbone=Qwen2.5-7B, Downstream Task Context=AG News, Safety Judge=Beaver-dam-7B2026.06 | 39.07 | — | |
| SaLoRABackbone=Qwen2.5-7B, Downstream Task Context=Average, Safety Judge=Beaver-dam-7B2026.06 | 39.21 | — | |
| SafeMERGEBackbone=Qwen2.5-7B, Downstream Task Context=BoolQ, Safety Judge=Beaver-dam-7B2026.06 | 39.64 | — | |
| SafeMERGEBackbone=Qwen2.5-7B, Downstream Task Context=MNLI, Safety Judge=Beaver-dam-7B2026.06 | 40.62 | — | |
| SafeMERGEBackbone=Qwen2.5-7B, Downstream Task Context=Average, Safety Judge=Beaver-dam-7B2026.06 | 40.67 | — | |
| SaLoRABackbone=Qwen2.5-7B, Downstream Task Context=MNLI, Safety Judge=Beaver-dam-7B2026.06 | 40.68 | — | |
| SafeMERGEBackbone=Qwen2.5-7B, Downstream Task Context=SST2, Safety Judge=Beaver-dam-7B2026.06 | 41.03 | — | |
| SafeMERGEBackbone=Qwen2.5-7B, Downstream Task Context=AG News, Safety Judge=Beaver-dam-7B2026.06 | 41.37 | — | |
| SafeLoRABackbone=Qwen2.5-7B, Downstream Task Context=MNLI, Safety Judge=Beaver-dam-7B2026.06 | 41.78 | — | |
| SafeLoRABackbone=Qwen2.5-7B, Downstream Task Context=SST2, Safety Judge=Beaver-dam-7B2026.06 | 42.41 | — | |
| SafeLoRABackbone=Qwen2.5-7B, Downstream Task Context=BoolQ, Safety Judge=Beaver-dam-7B2026.06 | 42.7 | — | |
| SafeLoRABackbone=Qwen2.5-7B, Downstream Task Context=Average, Safety Judge=Beaver-dam-7B2026.06 | 42.85 | — | |
| Fine-tunedBackbone=Qwen2.5-7B, Downstream Task Context=MNLI, Safety Judge=Beaver-dam-7B2026.06 | 43.39 | — | |
| SafeLoRABackbone=Qwen2.5-7B, Downstream Task Context=AG News, Safety Judge=Beaver-dam-7B2026.06 | 44.49 | — | |
| Fine-tunedBackbone=Qwen2.5-7B, Downstream Task Context=BoolQ, Safety Judge=Beaver-dam-7B2026.06 | 47.89 | — | |
| Fine-tunedBackbone=Qwen2.5-7B, Downstream Task Context=Average, Safety Judge=Beaver-dam-7B2026.06 | 49.58 | — | |
| SFTBackbone=Qwen-2.5-7B-Instruct2026.05 | 50.2 | 2.7 | |
| Fine-tunedBackbone=Qwen2.5-7B, Downstream Task Context=AG News, Safety Judge=Beaver-dam-7B2026.06 | 50.61 | — | |
| SafeInstrBackbone=Qwen-2.5-7B-Instruct2026.05 | 51.4 | 2.68 | |
| Fine-tunedBackbone=Qwen2.5-7B, Downstream Task Context=SST2, Safety Judge=Beaver-dam-7B2026.06 | 56.43 | — | |
| LisaTraining=Full fine-tuning, Backbone=SmolLM2-1.7B-Instruct2026.05 | 69 | 3.4 | |
| SFTTraining=Full fine-tuning, Backbone=SmolLM2-1.7B-Instruct2026.05 | 74 | 3.61 | |
| SFTModel=Qwen-3-8B2026.05 | 76.4 | 3.57 |