Safety evaluation on AdvBench 50 examples
100Safe Response RateDirect Instruction
Evaluation Results
| Method | Links | |
|---|---|---|
| Direct InstructionTarget Model=GPT-OSS 120b2025.09 | 100 | |
| GCGTarget Model=GPT-OSS 120b2025.09 | 100 | |
| FFATarget Model=GPT-OSS 120b2025.09 | 100 | |
| AIMTarget Model=GPT-OSS 120b2025.09 | 100 | |
| GCGTarget Model=Qwen32025.09 | 100 | |
| Direct InstructionTarget Model=Phi-3.52025.09 | 100 | |
| GCGTarget Model=Phi-3.52025.09 | 100 | |
| FFATarget Model=Phi-3.52025.09 | 100 | |
| Direct InstructionTarget Model=OLMoE2025.09 | 100 | |
| AIMTarget Model=OLMoE2025.09 | 100 | |
| Direct InstructionTarget Model=Qwen32025.09 | 98 | |
| GCGTarget Model=OLMoE2025.09 | 98 | |
| ArtPromptTarget Model=Qwen32025.09 | 96 | |
| AIMTarget Model=Phi-3.52025.09 | 96 | |
| SteerMoETarget Model=Phi-3.52025.09 | 94 | |
| FFATarget Model=OLMoE2025.09 | 92 | |
| SteerMoETarget Model=GPT-OSS 120b2025.09 | 90 | |
| SteerMoETarget Model=OLMoE2025.09 | 88 | |
| ArtPromptTarget Model=OLMoE2025.09 | 86 | |
| SteerMoE + FFATarget Model=OLMoE2025.09 | 70 | |
| SteerMoETarget Model=Qwen32025.09 | 60 | |
| ArtPromptTarget Model=GPT-OSS 120b2025.09 | 58 | |
| SteerMoE + FFATarget Model=Phi-3.52025.09 | 56 | |
| FFATarget Model=Qwen32025.09 | 48 | |
| SteerMoE + FFATarget Model=Qwen32025.09 | 48 | |
| ArtPromptTarget Model=Phi-3.52025.09 | 40 | |
| SteerMoE + AIMTarget Model=OLMoE2025.09 | 36 | |
| SteerMoE + FFATarget Model=GPT-OSS 120b2025.09 | 18 | |
| AIMTarget Model=Qwen32025.09 | 2 | |
| SteerMoE + AIMTarget Model=Qwen32025.09 | 2 | |
| SteerMoE + AIMTarget Model=GPT-OSS 120b2025.09 | 0 | |
| SteerMoE + AIMTarget Model=Phi-3.52025.09 | 0 |