Jailbreak Attack Success on SpatialJB Summary 1.0
76Acrostic ASRLlama4
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Llama4Guardrail=None2026.01 | 76 | 73.7 | 75.3 | 75.6 | 74.3 | 76.3 | 9.7 | 8.4 | 9.5 | |
| Grok4Guardrail=None2026.01 | 75 | 75 | 75 | 75 | 75 | 75 | 9.5 | 8.2 | 8.9 | |
| Deepseek-R1Guardrail=None2026.01 | 74 | 74.5 | 74.2 | 73.7 | 75 | 75.3 | 8.2 | 6.2 | 7.6 | |
| Deepseek-V3Guardrail=None2026.01 | 73.7 | 75.3 | 74.6 | 74.6 | 73.8 | 75.3 | 8.7 | 6.4 | 7.8 | |
| GPT4Guardrail=None2026.01 | 71.2 | 71.7 | 72.8 | 70.7 | 73 | 73.7 | 7.8 | 4.8 | 7 | |
| Gemini 2.5-ProGuardrail=None2026.01 | 70.3 | 68.5 | 67.3 | 69.8 | 69.8 | 70.7 | 6.4 | 4.5 | 7.9 | |
| Deepseek-V3Guardrail=OpenAI2026.01 | 68.8 | 66.5 | 68.8 | 58.7 | 69 | 67.5 | 6.2 | 4.5 | 5.8 | |
| Deepseek-V3Guardrail=Perspective2026.01 | 68.5 | 69.2 | 68 | 67.3 | 68.2 | 66.2 | 6 | 5.8 | 5.3 | |
| GPT4Guardrail=Llama Guard2026.01 | 68.2 | 67.3 | 68.2 | 65.8 | 68 | 69.2 | 5.3 | 3.9 | 5.1 | |
| Deepseek-V3Guardrail=Llama Guard2026.01 | 68.2 | 68.2 | 68.2 | 64.5 | 69.8 | 65.2 | 5.6 | 6 | 6.6 | |
| GPT4Guardrail=OpenAI2026.01 | 68 | 65 | 67.7 | 62.2 | 68.2 | 71.2 | 5.1 | 2.9 | 5.8 | |
| Llama4Guardrail=Perspective2026.01 | 68 | 67 | 67 | 66.5 | 68.5 | 67.2 | 6.6 | 5.3 | 4.5 | |
| Grok4Guardrail=Perspective2026.01 | 68 | 62.9 | 69.2 | 65.8 | 69.8 | 64.9 | 8.2 | 6.4 | 7.6 | |
| Deepseek-R1Guardrail=Llama Guard2026.01 | 67.3 | 67.3 | 68.2 | 67.3 | 69.2 | 67.5 | 6.4 | 4.3 | 5.1 | |
| Deepseek-R1Guardrail=OpenAI2026.01 | 66.5 | 60.5 | 61 | 56.2 | 62.2 | 58 | 5.8 | 4.3 | 4.5 | |
| Deepseek-R1Guardrail=Perspective2026.01 | 66.2 | 66.8 | 64.7 | 65.8 | 65.2 | 63.4 | 6.4 | 5 | 5.1 | |
| Grok4Guardrail=Llama Guard2026.01 | 65.8 | 67.7 | 65.5 | 67.3 | 70.3 | 66.2 | 6.6 | 7 | 7.6 | |
| Gemini 2.5-ProGuardrail=Llama Guard2026.01 | 64.7 | 59.8 | 54.8 | 61.3 | 61.5 | 64.2 | 5.3 | 3.3 | 6.4 | |
| Llama4Guardrail=Llama Guard2026.01 | 64.3 | 66.8 | 66.5 | 65 | 65.2 | 67.5 | 6.6 | 7 | 8.2 | |
| Grok4Guardrail=OpenAI2026.01 | 63.8 | 63.5 | 59 | 54.8 | 59.5 | 60 | 6.4 | 6 | 6.6 | |
| Llama4Guardrail=OpenAI2026.01 | 63 | 62.8 | 60.2 | 59.8 | 62.8 | 64.2 | 7.6 | 7.2 | 6.6 | |
| Gemini 2.5-ProGuardrail=Perspective2026.01 | 61.7 | 61.3 | 60.2 | 59.5 | 61.3 | 63.2 | 4.5 | 3.9 | 6 | |
| Gemini 2.5-ProGuardrail=OpenAI2026.01 | 57.8 | 56.8 | 56 | 55.3 | 59 | 55.9 | 4.5 | 3.5 | 6 | |
| GPT4Guardrail=Perspective2026.01 | 57.2 | 65.5 | 66.2 | 63.8 | 64.7 | 62.7 | 6.6 | 3.3 | 4.5 | |
| ClaudeGuardrail=None2026.01 | 51 | 49 | 52 | 47 | 59.8 | 57.4 | 6.6 | 4.3 | 5.6 | |
| ClaudeGuardrail=OpenAI2026.01 | 50.5 | 46.3 | 50.5 | 43 | 58 | 49 | 5.3 | 3.1 | 3.7 | |
| ClaudeGuardrail=Perspective2026.01 | 48.5 | 47.2 | 47.2 | 45.8 | 58.3 | 52.9 | 5.3 | 3.7 | 4.3 | |
| ClaudeGuardrail=Llama Guard2026.01 | 42.8 | 45.5 | 44.8 | 43.3 | 52 | 46 | 5.8 | 3.3 | 4.5 |