Agent Safety Reasoning on ASSEBench-Corrected
84.72AccuracyDRAFT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DRAFTBackbone=Qwen2.5-7B-Instruct2026.02 | 84.72 | 87.86 | 82.92 | 93.43 | |
| DRAFTBackbone=Qwen3-4B2026.02 | 84.17 | 87.25 | 93.33 | 91.55 | |
| DRAFTBackbone=Llama-3.1-8B-Instruct2026.02 | 83.89 | 87.34 | 81.63 | 93.9 | |
| ExtractorBackbone=Qwen2.5-7B-Instruct2026.02 | 83.61 | 87.31 | 80.56 | 95.31 | |
| SFTBackbone=Qwen3-4B2026.02 | 82.78 | 84.73 | 89.12 | 80.75 | |
| DRAFTBackbone=Qwen2.5-3B-Instruct2026.02 | 81.67 | 86.08 | 78.16 | 95.77 | |
| SFTBackbone=Llama-3.1-8B-Instruct2026.02 | 81.39 | 84.09 | 85.1 | 83.1 | |
| SFTBackbone=Qwen2.5-7B-Instruct2026.02 | 81.11 | 83.96 | 84.36 | 83.57 | |
| ExtractorBackbone=Qwen3-4B2026.02 | 80.56 | 85.04 | 78.04 | 93.43 | |
| DRAFTBackbone=Qwen2.5-1.5B-Instruct2026.02 | 80.28 | 84.67 | 78.42 | 92.02 | |
| ExtractorBackbone=Llama-3.1-8B-Instruct2026.02 | 78.61 | 84.19 | 74.82 | 96.24 | |
| ExtractorBackbone=Qwen2.5-3B-Instruct2026.02 | 78.06 | 83.64 | 74.81 | 94.84 | |
| SFTBackbone=Qwen2.5-1.5B-Instruct2026.02 | 74.17 | 80.25 | 73.26 | 88.73 | |
| ExtractorBackbone=Qwen2.5-1.5B-Instruct2026.02 | 73.61 | 81.19 | 70.21 | 96.24 | |
| SFTBackbone=Qwen2.5-3B-Instruct2026.02 | 73.61 | 75.2 | 84.71 | 67.61 | |
| VanillaBackbone=Qwen2.5-3B-Instruct2026.02 | 65.14 | 74.6 | 66.81 | 84.44 | |
| LoRABackbone=Qwen2.5-7B-Instruct2026.02 | 64.72 | 69.83 | 70.67 | 69.01 | |
| VanillaBackbone=Qwen2.5-7B-Instruct2026.02 | 63.38 | 69.34 | 70.37 | 68.34 | |
| LoRABackbone=Qwen2.5-3B-Instruct2026.02 | 61.11 | 70.83 | 63.67 | 79.81 | |
| VanillaBackbone=Qwen3-4B2026.02 | 60.8 | 67.65 | 67.65 | 67.65 | |
| LoRABackbone=Qwen3-4B2026.02 | 60.56 | 66.51 | 66.82 | 66.2 | |
| LoRABackbone=Llama-3.1-8B-Instruct2026.02 | 57.5 | 51.43 | 79.41 | 38.03 | |
| VanillaBackbone=Llama-3.1-8B-Instruct2026.02 | 57.1 | 53.79 | 77.42 | 41.21 | |
| LoRABackbone=Qwen2.5-1.5B-Instruct2026.02 | 35.01 | 19.86 | 36.71 | 13.62 | |
| VanillaBackbone=Qwen2.5-1.5B-Instruct2026.02 | 32.11 | 18.04 | 33.65 | 12.33 |