Goal Hijacking on Target Response Dataset Llama-2 targets (test)
0ASR (threatening)MAC-hijacking
Evaluation results
| Method | Links | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MAC-hijackingBase Model=llama-22024.05 | 0 | 0 | 0.7 | 0.2 | 0.2 | 0.8 | 0 | 0.1 | 1.6 | 0 | 0.36 | 546 | 490 | 362 | 479 | 344 | 309 | 758 | 299 | 518 | 600 | 470.5 | |
| AutoDAN-hijackingBase Model=llama-22024.05 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 500 | 500 | 500 | 500 | 500 | 500 | 500 | 500 | 500 | 500 | 500 | |
| AmpleGCG-hijackingBase Model=llama-22024.05 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | — | — | — | — | — | — | — | — | — | — | — | |
| HouYiBase Model=llama-22024.05 | 0 | 0 | 0.6 | 0.2 | 0 | 0.1 | 2.8 | 0 | 0 | 0 | 0.37 | — | — | — | — | — | — | — | — | — | — | — | |
| Tensor TrustBase Model=llama-22024.05 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | — | — | — | — | — | — | — | — | — | — | — | |
| GCG-hijackingBase Model=llama-22024.05 | 0.2 | 0 | 0.1 | 7.1 | 0.2 | 0 | 0.5 | 0 | 0.3 | 0 | 0.84 | 303 | 567 | 267 | 435 | 450 | 205 | 578 | 909 | 445 | 461 | 462 | |
| M-GCGBase Model=llama-22024.05 | 24.8 | 0 | 79.8 | 0 | 93.6 | 88.6 | 94.3 | 0 | 92.8 | 68.7 | 54.26 | 25,000 | 25,000 | 25,000 | 25,000 | 25,000 | 25,000 | 25,000 | 25,000 | 25,000 | 25,000 | 25,000 | |
| I-UGHBase Model=llama-22024.05 | 92.5 | 84.2 | 86.3 | 82.8 | 88.7 | 88.9 | 70.8 | 79.7 | 88.8 | 91.9 | 85.5 | 11,280 | 4,672 | 2,844 | 7,294 | 11,722 | 2,795 | 19,444 | 9,726 | 2,340 | 4,014 | 7,613.1 | |
| POUGHBase Model=llama-22024.05 | 92.6 | 93.5 | 94.4 | 97.3 | 92.8 | 92 | 97.1 | 82.9 | 98.7 | 92.8 | 93.41 | 2,092 | 23,478 | 2,306 | 6,105 | 3,049 | 3,406 | 3,600 | 2,589 | 6,109 | 3,528 | 5,626.2 |