Adversarial Attack on HarmBench (HB) and JailbreakBench (JBB) (held-out prompts)
72.1ASR (%)PGD-10
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PGD-10Backbone=Qwen2.5-3B, Gradient Calculation=Exact gradient2026.05 | 72.1 | 0.6 | 0.46 | |
| PGD-5Backbone=Qwen2.5-3B, Gradient Calculation=Exact gradient2026.05 | 59.9 | 1.3 | 0.76 | |
| PGD-10Backbone=Llama-2-7B, Gradient Calculation=Exact gradient2026.05 | 49.3 | 0.7 | 0.36 | |
| PGD-5Backbone=Llama-2-7B, Gradient Calculation=Exact gradient2026.05 | 41.4 | 1.5 | 0.61 | |
| PGD-2Backbone=Llama-2-7B, Gradient Calculation=Exact gradient2026.05 | 39.3 | 3.7 | 1.44 | |
| PGD-10Backbone=Qwen3-4B, Gradient Calculation=Exact gradient2026.05 | 35.6 | 0.5 | 0.18 | |
| PGD-5Backbone=Qwen3-4B, Gradient Calculation=Exact gradient2026.05 | 32.4 | 1 | 0.33 | |
| PGD-2Backbone=Qwen2.5-3B, Gradient Calculation=Exact gradient2026.05 | 31 | 3.1 | 0.97 | |
| FGSMBackbone=Qwen2.5-3B, Gradient Calculation=Exact gradient2026.05 | 24.5 | 6.3 | 1.55 | |
| RS-FGSMBackbone=Qwen2.5-3B, Gradient Calculation=Exact gradient2026.05 | 22.8 | 6.3 | 1.43 | |
| PGD-10Backbone=Qwen2.5-3B, Gradient Calculation=GPR (predicted)2026.05 | 18.7 | 7.7 | 1.43 | |
| FGSMBackbone=Llama-2-7B, Gradient Calculation=Exact gradient2026.05 | 18.6 | 7.4 | 1.37 | |
| RS-FGSMBackbone=Llama-2-7B, Gradient Calculation=Exact gradient2026.05 | 18.6 | 7.3 | 1.35 | |
| PGD-5Backbone=Qwen2.5-3B, Gradient Calculation=GPR (predicted)2026.05 | 17.3 | 15.2 | 2.63 | |
| PGD-2Backbone=Qwen3-4B, Gradient Calculation=Exact gradient2026.05 | 16.5 | 2.6 | 0.42 | |
| RS-FGSMBackbone=Llama-2-7B, Gradient Calculation=GPR (predicted)2026.05 | 16.4 | 47.1 | 7.74 | |
| PGD-5Backbone=Llama-2-7B, Gradient Calculation=GPR (predicted)2026.05 | 15.7 | 9.7 | 1.52 | |
| FGSMBackbone=Llama-2-7B, Gradient Calculation=GPR (predicted)2026.05 | 15.4 | 47.4 | 7.28 | |
| PGD-10Backbone=Llama-2-7B, Gradient Calculation=GPR (predicted)2026.05 | 15.4 | 4.9 | 0.75 | |
| RS-FGSMBackbone=Qwen2.5-3B, Gradient Calculation=GPR (predicted)2026.05 | 13.6 | 72.3 | 9.84 | |
| PGD-2Backbone=Llama-2-7B, Gradient Calculation=GPR (predicted)2026.05 | 13.6 | 23.9 | 3.25 | |
| FGSMBackbone=Qwen3-4B, Gradient Calculation=Exact gradient2026.05 | 12.6 | 5.1 | 0.65 | |
| FGSMBackbone=Qwen2.5-3B, Gradient Calculation=GPR (predicted)2026.05 | 11.6 | 73.4 | 8.49 | |
| RS-FGSMBackbone=Qwen3-4B, Gradient Calculation=Exact gradient2026.05 | 11 | 5.1 | 0.56 | |
| FGMBackbone=Llama-2-7B, Gradient Calculation=Exact gradient2026.05 | 10 | 7.4 | 0.74 | |
| FGMBackbone=Qwen3-4B, Gradient Calculation=Exact gradient2026.05 | 9.7 | 5.1 | 0.5 | |
| RS-FGSMBackbone=Qwen3-4B, Gradient Calculation=GPR (predicted)2026.05 | 8.1 | 60.3 | 4.88 | |
| PGD-2Backbone=Qwen2.5-3B, Gradient Calculation=GPR (predicted)2026.05 | 7.5 | 37.2 | 2.78 | |
| FGSMBackbone=Qwen3-4B, Gradient Calculation=GPR (predicted)2026.05 | 6.8 | 60.5 | 4.11 | |
| PGD-2Backbone=Qwen3-4B, Gradient Calculation=GPR (predicted)2026.05 | 6.5 | 30.8 | 2 | |
| PGD-5Backbone=Qwen3-4B, Gradient Calculation=GPR (predicted)2026.05 | 6.5 | 12.5 | 0.81 | |
| FGMBackbone=Qwen2.5-3B, Gradient Calculation=Exact gradient2026.05 | 6.1 | 6.1 | 0.38 | |
| FGMBackbone=Qwen3-4B, Gradient Calculation=GPR (predicted)2026.05 | 4.2 | 59.6 | 2.51 | |
| PGD-10Backbone=Qwen3-4B, Gradient Calculation=GPR (predicted)2026.05 | 3.9 | 6.3 | 0.25 | |
| FGMBackbone=Qwen2.5-3B, Gradient Calculation=GPR (predicted)2026.05 | 3.4 | 71.4 | 2.43 | |
| FGMBackbone=Llama-2-7B, Gradient Calculation=GPR (predicted)2026.05 | 2.1 | 47 | 1.01 |