Trajectory-safety classification on ATBench-C
77.8AccuracyQwen2-7B-Instruct + FATE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen2-7B-Instruct + FATEModel Type=Ours2026.05 | 77.8 | 80.5 | 78.6 | 79.5 | |
| Gemini-1.5-FlashModel Type=Closed-source models2026.05 | 76.4 | 79.3 | 71 | 74.9 | |
| Gemini-1.5-ProModel Type=Closed-source models2026.05 | 75.5 | 76.1 | 73.8 | 75 | |
| GPT-4Model Type=Closed-source models2026.05 | 73.7 | 68.5 | 87.1 | 76.7 | |
| GPT-4oModel Type=Closed-source models2026.05 | 69 | 65.6 | 79.3 | 71.8 | |
| Qwen2.5-72B-InstructModel Type=Open-source models2026.05 | 66.8 | 65.5 | 70.2 | 67.8 | |
| AgentDoG-Qwen2-7BModel Type=Guard models2026.05 | 64 | 59.2 | 88.9 | 71.1 | |
| ShieldAgentModel Type=Guard models2026.05 | 62.5 | 58 | 81.4 | 67.7 | |
| Qwen2.5-32B-InstructModel Type=Open-source models2026.05 | 59.2 | 58.2 | 63.8 | 60.8 | |
| LlamaGuard2-8BModel Type=Guard models2026.05 | 58.1 | 63.8 | 30.9 | 41.7 | |
| QwQ-32B-PreviewModel Type=Open-source models2026.05 | 57.7 | 81.9 | 19.1 | 31 | |
| Qwen2-7B-InstructModel Type=Open-source models2026.05 | 55.7 | 77.6 | 15.3 | 25.5 | |
| Qwen2.5-7B-InstructModel Type=Open-source models2026.05 | 53.4 | 73.8 | 9.7 | 17.1 | |
| LlamaGuard3-8BModel Type=Guard models2026.05 | 53.1 | 85.7 | 3.8 | 7.3 | |
| Qwen2-7BModel Type=Open-source models2026.05 | 52.6 | 78 | 6.4 | 11.9 | |
| Qwen2-GuardModel Type=Guard models2026.05 | 51.5 | 40 | 0.4 | 0.8 | |
| Qwen2.5-7BModel Type=Open-source models2026.05 | 45.9 | 41.2 | 20.7 | 27.6 | |
| Llama-3.1-8B-InstructModel Type=Open-source models2026.05 | 45.3 | 47.3 | 89.5 | 61.9 |