Jailbreak evaluation on Red Queen SET
23Passed CountQwen 3.5 35B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen 3.5 35BExecuted with ALM=true, Temperature=1.0, Max Tokens=7680, Top K=20, Top P=0.9, Repeat Penalty=1.1, Presence Penalty=1.52026.04 | 23 | 2 | 8 | 2 | |
| Llama 3.2 3BExecuted with ALM=false, Temperature=0.8, Max Tokens=768, Top K=40, Top P=0.9, Repeat Penalty=1.12026.04 | 23 | 2 | 8 | 2 | |
| Nemotron 3 Super 120BExecuted with ALM=true, Temperature=1.0, Max Tokens=768, Top K=40, Top P=0.95, Repeat Penalty=1.12026.04 | 22 | 3 | 12 | 4 | |
| Llama 3.1 8BExecuted with ALM=false, Temperature=0.8, Max Tokens=768, Top K=40, Top P=0.9, Repeat Penalty=1.12026.04 | 21 | 4 | 16 | 6 | |
| Llama 3.3 70BExecuted with ALM=false, Temperature=0.8, Max Tokens=768, Top K=40, Top P=0.9, Repeat Penalty=1.12026.04 | 21 | 4 | 16 | 6 | |
| Qwen 3.5 35BExecuted with ALM=false, Temperature=1.0, Max Tokens=7680, Top K=20, Top P=0.9, Repeat Penalty=1.1, Presence Penalty=1.52026.04 | 20 | 5 | 20 | 9 | |
| Nemotron 3 Super 120BExecuted with ALM=false, Temperature=1.0, Max Tokens=768, Top K=40, Top P=0.95, Repeat Penalty=1.12026.04 | 20 | 5 | 20 | 9 | |
| Qwen 3 32BExecuted with ALM=false, Temperature=0.6, Max Tokens=7680, Top K=20, Top P=0.9, Repeat Penalty=1.02026.04 | 17 | 8 | 32 | 17 | |
| Ministral 3 14BExecuted with ALM=false, Temperature=0.15, Max Tokens=768, Top K=40, Top P=0.9, Repeat Penalty=1.12026.04 | 16 | 9 | 36 | 20 | |
| Mistral 3.2 24BExecuted with ALM=false, Temperature=0.15, Max Tokens=768, Top K=40, Top P=0.9, Repeat Penalty=1.12026.04 | 16 | 9 | 36 | 20 | |
| Mistral 3.2 24BExecuted with ALM=true, Temperature=0.15, Max Tokens=768, Top K=40, Top P=0.9, Repeat Penalty=1.12026.04 | 15 | 10 | 40 | 23 | |
| Nemotron 3 Nano 30BExecuted with ALM=true, Temperature=1.0, Max Tokens=768, Top K=40, Top P=1.0, Repeat Penalty=1.12026.04 | 15 | 10 | 40 | 23 | |
| Nemotron 3 Nano 30BExecuted with ALM=false, Temperature=1.0, Max Tokens=768, Top K=40, Top P=1.0, Repeat Penalty=1.12026.04 | 12 | 13 | 52 | 33 | |
| Llama 3.3 70BExecuted with ALM=true, Temperature=0.8, Max Tokens=768, Top K=40, Top P=0.9, Repeat Penalty=1.12026.04 | 9 | 16 | 64 | 45 | |
| Llama 3.1 8BExecuted with ALM=true, Temperature=0.8, Max Tokens=768, Top K=40, Top P=0.9, Repeat Penalty=1.12026.04 | 8 | 17 | 68 | 48 | |
| Llama 3.2 3BExecuted with ALM=true, Temperature=0.8, Max Tokens=768, Top K=40, Top P=0.9, Repeat Penalty=1.12026.04 | 8 | 17 | 68 | 48 | |
| Qwen 3 32BExecuted with ALM=true, Temperature=0.6, Max Tokens=7680, Top K=20, Top P=0.9, Repeat Penalty=1.02026.04 | 8 | 17 | 68 | 48 | |
| Ministral 3 14BExecuted with ALM=true, Temperature=0.15, Max Tokens=768, Top K=40, Top P=0.9, Repeat Penalty=1.12026.04 | 4 | 21 | 84 | 65 |