Red-teaming Attack Success Rate on JailbreakBench (test)
82ASR (Vicuna)None
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| NoneAttack=PAIR2024.01 | 82 | 4 | 68 | 2 | 76 | 50 | |
| Perplexity FilterAttack=PAIR2024.01 | 81 | 4 | 66 | 2 | 15 | 43 | |
| NoneAttack=JBC2024.01 | 79 | 0 | 58 | 1 | 0 | 0 | |
| Perplexity FilterAttack=JBC2024.01 | 79 | 0 | 58 | 1 | 0 | 0 | |
| SmoothLLMAttack=JBC2024.01 | 64 | 0 | 44 | 0 | 0 | 0 | |
| NoneAttack=GCG2024.01 | 58 | 2 | 11 | 0 | 34 | 1 | |
| SmoothLLMAttack=PAIR2024.01 | 47 | 1 | 36 | 1 | 12 | 25 | |
| Few-ShotAttack=JBC2024.01 | 41 | 0 | 50 | 1 | 0 | 0 | |
| Few-ShotAttack=PAIR2024.01 | 27 | 6 | 16 | 1 | 8 | 10 | |
| RephrasingAttack=PAIR2024.01 | 25 | 4 | 13 | 2 | 22 | 35 | |
| RPOAttack=JBC2024.01 | 20 | 0 | 45 | 0 | 0 | 0 | |
| RPOAttack=PAIR2024.01 | 16 | 0 | 4 | 1 | 6 | 6 | |
| RephrasingAttack=JBC2024.01 | 15 | 0 | 47 | 1 | 1 | 1 | |
| RephrasingAttack=GCG2024.01 | 4 | 0 | 7 | 0 | 4 | 1 | |
| SmoothLLMAttack=GCG2024.01 | 1 | 1 | 2 | 0 | 1 | 3 | |
| Perplexity FilterAttack=GCG2024.01 | 1 | 0 | 0 | 0 | 1 | 0 | |
| Few-ShotAttack=GCG2024.01 | 1 | 0 | 1 | 0 | 0 | 0 | |
| RPOAttack=GCG2024.01 | 0 | 0 | 0 | 0 | 0 | 0 |