Jailbreak Attack on HADES (test)
96Self-harm Success RateGAMBIT
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GAMBITModel=Gemini 2.5 Flash, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 96 | 94.67 | 94.67 | 80.67 | 94.67 | 92.13 | |
| GAMBITModel=GLM-4.1V-Thinking, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 94 | 91.33 | 94 | 75.33 | 92.67 | 89.47 | |
| GAMBITModel=QvQ-Max, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 92 | 96 | 95.33 | 78.67 | 94 | 91.2 | |
| VisCRAModel=Gemini 2.5 Flash, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 62.67 | 70.67 | 71.33 | 44.67 | 80.67 | 66 | |
| VisCRAModel=QvQ-Max, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 59.33 | 78 | 76 | 41.33 | 76.67 | 66.27 | |
| HADES AttackModel=GLM-4.1V-Thinking, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 51.33 | 47.33 | 62 | 40 | 78 | 55.73 | |
| SI-AModel=Gemini 2.5 Flash, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 49.33 | 65.33 | 74.67 | 44 | 77.33 | 62.13 | |
| SI-AModel=GLM-4.1V-Thinking, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 46 | 50.67 | 62.67 | 37.33 | 76 | 54.53 | |
| GAMBITModel=OpenAI o4-mini, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 32 | 32.67 | 28.67 | 27.33 | 36 | 31.33 | |
| SI-AModel=QvQ-Max, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 29.33 | 45.33 | 64 | 24.67 | 76 | 47.87 | |
| HADES AttackModel=QvQ-Max, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 19.33 | 48.67 | 45.33 | 7.33 | 57.33 | 35.6 | |
| SI-AModel=OpenAI o4-mini, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 10 | 8 | 10.67 | 11.33 | 6.67 | 9.33 | |
| HADES AttackModel=Gemini 2.5 Flash, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 8 | 16.67 | 29.33 | 2 | 18 | 14.8 | |
| VisCRAModel=OpenAI o4-mini, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 4.67 | 9.33 | 21.33 | 12 | 11.33 | 11.73 | |
| HADES AttackModel=OpenAI o4-mini, Pass@k=Pass@5, Evaluator=Llama-Guard-32026.01 | 0.67 | 0.67 | 2 | 0 | 0 | 0.67 |