Safety Evaluation on MEMEVOBENCH Workflow Style 1.0 (test)
88ASR (Rd.1)Llama-3.3-70B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Llama-3.3-70BPrompting Strategy=+SafePrompt, User Feedback=w/feedback2026.04 | 88 | 89.2 | 92.8 | |
| Llama-3.3-70BPrompting Strategy=Vanilla, User Feedback=w/feedback2026.04 | 86.7 | 90.4 | 94 | |
| Llama-3.3-70BPrompting Strategy=Vanilla, User Feedback=None2026.04 | 85.4 | 87.8 | 90.2 | |
| Llama-3.3-70BPrompting Strategy=+SafePrompt, User Feedback=None2026.04 | 84.3 | 88 | 90.4 | |
| Llama-3.3-70BPrompting Strategy=+ModTool, User Feedback=w/feedback2026.04 | 83.1 | 86.7 | 88 | |
| Llama-3.3-70BPrompting Strategy=+ModTool, User Feedback=None2026.04 | 81.9 | 88 | 90.4 | |
| GPT-4oPrompting Strategy=Vanilla, User Feedback=None2026.04 | 81.9 | 85.5 | 91.6 | |
| GPT-4oPrompting Strategy=+SafePrompt, User Feedback=w/feedback2026.04 | 79.5 | 89.2 | 90.4 | |
| Qwen3-32BPrompting Strategy=Vanilla, User Feedback=w/feedback2026.04 | 78.3 | 85.5 | 91.6 | |
| Qwen3-Next-80BPrompting Strategy=Vanilla, User Feedback=None2026.04 | 78.3 | 80.7 | 84.3 | |
| DeepSeek-V3.2Prompting Strategy=+SafePrompt, User Feedback=w/feedback2026.04 | 78.3 | 83.1 | 89.1 | |
| GPT-4oPrompting Strategy=+ModTool, User Feedback=None2026.04 | 78.3 | 85.5 | 86.7 | |
| Qwen3-32BPrompting Strategy=Vanilla, User Feedback=None2026.04 | 77.1 | 86.9 | 89.2 | |
| Qwen3-Next-80BPrompting Strategy=Vanilla, User Feedback=w/feedback2026.04 | 77.1 | 81.9 | 89.2 | |
| Qwen3-235B-A22BPrompting Strategy=Vanilla, User Feedback=None2026.04 | 77.1 | 80.7 | 83.1 | |
| DeepSeek-V3.2Prompting Strategy=+ModTool, User Feedback=None2026.04 | 77.1 | 80.7 | 73.5 | |
| DeepSeek-V3.2Prompting Strategy=+ModTool, User Feedback=w/feedback2026.04 | 77.1 | 84.3 | 85.5 | |
| GPT-4oPrompting Strategy=+ModTool, User Feedback=w/feedback2026.04 | 77.1 | 86.7 | 89.2 | |
| DeepSeek-V3.2Prompting Strategy=Vanilla, User Feedback=None2026.04 | 75.9 | 86.7 | 84.3 | |
| GPT-4oPrompting Strategy=+SafePrompt, User Feedback=None2026.04 | 75.9 | 84.3 | 89.2 | |
| Qwen3-32BPrompting Strategy=+ModTool, User Feedback=w/feedback2026.04 | 75.6 | 84.1 | 81.7 | |
| Qwen3-235B-A22BPrompting Strategy=+ModTool, User Feedback=w/feedback2026.04 | 75.6 | 84.1 | 81.7 | |
| Qwen3-32BPrompting Strategy=+SafePrompt, User Feedback=w/feedback2026.04 | 74.7 | 86.7 | 92.8 | |
| Qwen3-Next-80BPrompting Strategy=+SafePrompt, User Feedback=None2026.04 | 74.7 | 79.5 | 84.3 | |
| Qwen3-235B-A22BPrompting Strategy=+SafePrompt, User Feedback=None2026.04 | 74.7 | 79.5 | 84.3 | |
| Qwen3-235B-A22BPrompting Strategy=Vanilla, User Feedback=w/feedback2026.04 | 74.7 | 85.5 | 90.4 | |
| Gemini-2.5-ProPrompting Strategy=Vanilla, User Feedback=None2026.04 | 74.7 | 79.5 | 84.3 | |
| Gemini-2.5-ProPrompting Strategy=Vanilla, User Feedback=w/feedback2026.04 | 74.7 | 84.3 | 85.5 | |
| GPT-4oPrompting Strategy=Vanilla, User Feedback=w/feedback2026.04 | 74.4 | 86.6 | 91.5 | |
| Qwen3-32BPrompting Strategy=+SafePrompt, User Feedback=None2026.04 | 73.5 | 86.7 | 89.2 | |
| Qwen3-Next-80BPrompting Strategy=+SafePrompt, User Feedback=w/feedback2026.04 | 73.5 | 81.9 | 89.2 | |
| Qwen3-235B-A22BPrompting Strategy=+SafePrompt, User Feedback=w/feedback2026.04 | 73.5 | 85.5 | 85.5 | |
| DeepSeek-V3.2Prompting Strategy=Vanilla, User Feedback=w/feedback2026.04 | 73.5 | 92.8 | 94 | |
| Gemini-2.5-ProPrompting Strategy=+SafePrompt, User Feedback=w/feedback2026.04 | 72.3 | 80.7 | 83.1 | |
| Claude-3.7-SonnetPrompting Strategy=+SafePrompt, User Feedback=w/feedback2026.04 | 72.3 | 80.7 | 83.1 | |
| DeepSeek-V3.2Prompting Strategy=+SafePrompt, User Feedback=None2026.04 | 70.7 | 80.5 | 81.7 | |
| Qwen3-Next-80BPrompting Strategy=+ModTool, User Feedback=w/feedback2026.04 | 69.9 | 80.7 | 85.5 | |
| Qwen3-235B-A22BPrompting Strategy=+ModTool, User Feedback=None2026.04 | 69.5 | 72 | 70 | |
| Qwen3-32BPrompting Strategy=+ModTool, User Feedback=None2026.04 | 68.7 | 79.5 | 80.7 | |
| Qwen3-Next-80BPrompting Strategy=+ModTool, User Feedback=None2026.04 | 68.7 | 77.1 | 78.3 | |
| Gemini-2.5-ProPrompting Strategy=+SafePrompt, User Feedback=None2026.04 | 67.5 | 78.3 | 79.5 | |
| Claude-3.7-SonnetPrompting Strategy=+SafePrompt, User Feedback=None2026.04 | 67.5 | 78.3 | 79.5 | |
| Claude-3.7-SonnetPrompting Strategy=Vanilla, User Feedback=w/feedback2026.04 | 66.3 | 81.3 | 87.5 | |
| GPT-5Prompting Strategy=Vanilla, User Feedback=None2026.04 | 65.4 | 70.5 | 69.2 | |
| Claude-3.7-SonnetPrompting Strategy=+ModTool, User Feedback=w/feedback2026.04 | 65.1 | 74.7 | 77.1 | |
| GPT-5Prompting Strategy=Vanilla, User Feedback=w/feedback2026.04 | 64.9 | 81.8 | 87 | |
| Claude-3.7-SonnetPrompting Strategy=Vanilla, User Feedback=None2026.04 | 64.2 | 76.5 | 82.7 | |
| Claude-3.7-SonnetPrompting Strategy=+ModTool, User Feedback=None2026.04 | 63.9 | 69.9 | 75.9 | |
| GPT-5Prompting Strategy=+SafePrompt, User Feedback=None2026.04 | 62 | 70 | 72.2 | |
| Gemini-2.5-ProPrompting Strategy=+ModTool, User Feedback=w/feedback2026.04 | 57.8 | 80 | 74.7 | |
| GPT-5Prompting Strategy=+ModTool, User Feedback=w/feedback2026.04 | 55.6 | 76.5 | 74.1 | |
| GPT-5Prompting Strategy=+ModTool, User Feedback=None2026.04 | 54.3 | 63 | 63 | |
| Gemini-2.5-ProPrompting Strategy=+ModTool, User Feedback=None2026.04 | 54.2 | 57.8 | 62.7 | |
| GPT-5Prompting Strategy=+SafePrompt, User Feedback=w/feedback2026.04 | 49.4 | 77.2 | 77.2 |