Jailbreak Robustness on JBB-Behaviors
0.3ASR (PAIR, Guardrail Model)RA (ours)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RA (ours)Base Model Architecture=LLaDA2025.10 | 0.3 | 10 | 45 | 60 | 29.3 | 67.3 | |
| MOSABase Model Architecture=LLaDA2025.10 | 1 | 27.7 | 65 | 56 | 54.3 | 74 | |
| RA (ours)Base Model Architecture=LLaDA1.52025.10 | 1.3 | 21 | 45 | 56.7 | 38.7 | 58.3 | |
| MOSABase Model Architecture=LLaDA1.52025.10 | 2 | 27.3 | 67.7 | 56.7 | 57 | 74.3 | |
| RA w/o inter (ablation)Base Model Architecture=LLaDA1.52025.10 | 2.3 | 36 | 68 | 62.5 | 53.7 | 72.3 | |
| DPOBase Model Architecture=LLaDA2025.10 | 2.7 | 33.3 | 69 | 57.3 | 60.7 | 84.7 | |
| DPOBase Model Architecture=LLaDA1.52025.10 | 3 | 37.7 | 70.3 | 61.3 | 66.7 | 83.7 | |
| SFTBase Model Architecture=LLaDA2025.10 | 4 | 39.3 | 68.7 | 56 | 70 | 85.3 | |
| RA w/o inter (ablation)Base Model Architecture=LLaDA2025.10 | 4 | 25.7 | 68.3 | 60.3 | 52.3 | 68.7 | |
| OriginalBase Model Architecture=LLaDA1.52025.10 | 4.7 | 44.7 | 74.3 | 63 | 68.3 | 91.7 | |
| OriginalBase Model Architecture=LLaDA2025.10 | 5 | 41.7 | 77.3 | 65 | 68 | 89.7 | |
| SFTBase Model Architecture=LLaDA1.52025.10 | 5.3 | 46.3 | 71 | 54.7 | 67.7 | 80.7 | |
| RA (ours)Base Model Architecture=MMaDA2025.10 | 13 | 42.3 | 50 | 65 | 65.3 | 90.7 | |
| MOSABase Model Architecture=MMaDA2025.10 | 17.7 | 46.3 | 68.3 | 65.7 | 55.3 | 83.7 | |
| RA w/o inter (ablation)Base Model Architecture=MMaDA2025.10 | 32 | 52.4 | 60 | 85 | 66.3 | 86.7 | |
| DPOBase Model Architecture=MMaDA2025.10 | 37.5 | 60 | 79 | 65.7 | 61.3 | 85.7 | |
| SFTBase Model Architecture=MMaDA2025.10 | 47.7 | 70.3 | 86.5 | 78.5 | 57 | 71.3 | |
| OriginalBase Model Architecture=MMaDA2025.10 | 55 | 81 | 81.7 | 85.7 | 64 | 88.3 |