Adversarial Robustness on HarmBench (DirReq, GCG, A.DAN, A.Pmpt, PAIR, TAP)
17DirReqBase (8B Instruct)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Base (8B Instruct)Base Model=Llama 32026.05 | 17 | 27 | 12 | 18 | 25 | 26 | |
| CATBase Model=Llama 22026.05 | 9 | 16 | 5 | 14 | 37 | 34 | |
| LPABase Model=Llama 32026.05 | 5 | 0 | 0 | 0 | 3 | 6 | |
| Base (7B chat-hf)Base Model=Llama 22026.05 | 3 | 40 | 7 | 17 | 15 | 21 | |
| LPABase Model=Llama 22026.05 | 3 | 34 | 6 | 12 | 11 | 20 | |
| CATBase Model=Llama 32026.05 | 1 | 1 | 0 | 3 | 26 | 19 | |
| LPA-overfitBase Model=Llama 32026.05 | 1 | 0 | 0 | 0 | 3 | 3 | |
| LATBase Model=Llama 32026.05 | 0 | 0 | 0 | 0 | 2 | 6 | |
| LATBase Model=Llama 22026.05 | 0 | 0 | 0 | 0 | 0 | 1 | |
| LPA-overfitBase Model=Llama 22026.05 | 0 | 16 | 1 | 7 | 6 | 15 |