Attack Success Rate on HEx-PHI
0Attack Success RateDataShield
Evaluation Results
| Method | Links | |
|---|---|---|
| DataShieldModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 0 | |
| LARFModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 0 | |
| DataShieldModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 0 | |
| LARFModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 0.34 | |
| DataShieldModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 0.34 | |
| Bi-AnchorModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 0.69 | |
| LARFModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 0.75 | |
| Bi-AnchorModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 1.03 | |
| DataShieldModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 1.38 | |
| DataShieldModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 1.38 | |
| LARFModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 2.07 | |
| Base2025.10 | 3 | |
| LARFModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 3.1 | |
| Bi-AnchorModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 3.45 | |
| GuardModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 4.14 | |
| GuardModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 4.83 | |
| RandomModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 5.17 | |
| RandomModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 5.17 | |
| InstructModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 5.86 | |
| InstructModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 5.86 | |
| SEALModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 6.55 | |
| SEALModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 6.9 | |
| SEALModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 6.9 | |
| SEALModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 7.24 | |
| LARFModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 7.59 | |
| InstructModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 7.93 | |
| InstructModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 7.93 | |
| DataShieldModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 8.28 | |
| RandomModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 9.31 | |
| RandomModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 9.31 | |
| RandomModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 9.66 | |
| RandomModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 9.66 | |
| InstructModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 10.69 | |
| InstructModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 10.69 | |
| Bi-AnchorModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 12.07 | |
| GuardModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 12.07 | |
| GradSafeModel=Llama3, Fine-tuning Dataset=Alpaca2026.05 | 13.45 | |
| GradSafeModel=Qwen2.5, Fine-tuning Dataset=Alpaca2026.05 | 13.79 | |
| GradSafeModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 15.52 | |
| GuardModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 23.45 | |
| Bi-AnchorModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 24.48 | |
| GuardModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 24.83 | |
| PatcherBase Model=Qwen2.5-1.5B2026.06 | 26.5 | |
| GradSafeModel=Llama3.1, Fine-tuning Dataset=Alpaca2026.05 | 29.31 | |
| PatcherBase Model=Qwen3-4B2026.06 | 31 | |
| Bi-AnchorModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 34.83 | |
| GradSafeModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 36.55 | |
| SEALModel=Llama3.1, Fine-tuning Dataset=Dolly2026.05 | 38.62 | |
| GradSafeModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 42.07 | |
| GuardModel=Llama3, Fine-tuning Dataset=Dolly2026.05 | 44.48 | |
| SEALModel=Qwen2.5, Fine-tuning Dataset=Dolly2026.05 | 51.03 | |
| PatcherBase Model=Llama3-8B2026.06 | 55.5 | |
| SFTBase Model=Qwen2.5-1.5B2026.06 | 58.6 | |
| BoosterBase Model=Qwen3-4B2026.06 | 63.3 | |
| BoosterBase Model=Qwen2.5-1.5B2026.06 | 63.9 | |
| VaccineBase Model=Qwen3-4B2026.06 | 66.3 | |
| SFTBase Model=Qwen3-4B2026.06 | 67.2 | |
| VaccineBase Model=Qwen2.5-1.5B2026.06 | 71.7 | |
| SFT2025.10 | 79.33 | |
| SFTBase Model=Llama3-8B2026.06 | 91.3 | |
| VaccineBase Model=Llama3-8B2026.06 | 92.8 | |
| BoosterBase Model=Llama3-8B2026.06 | 93 | |
| RLVR2025.10 | 94.67 |