Tool-Using Agent Safety on MT-AgentRisk 1.0 (test)
27.14Filesystem ASRGPT-5.2
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-5.2Setting=Single2026.02 | 27.14 | 71.43 | 22.14 | 67.14 | 25.71 | 60 | 22.86 | 71.43 | 60 | 40 | 25.48 | 66.3 | — | — | |
| Claude-4.5-SonnetSetting=Single2026.02 | 37.14 | 55.74 | 57.86 | 27.14 | 28.57 | 58.57 | 48.57 | 45.71 | 20 | 73.3 | 44.66 | 43.56 | — | — | |
| GPT-5.2Setting=Multi2026.02 | 51 | 40 | 34.29 | 50.71 | 37.14 | 54.29 | 31.43 | 67.14 | 86.67 | 0 | 39.73 | 50.41 | 55.9 | -24 | |
| Qwen3-CoderSetting=Single2026.02 | 55.71 | 32.86 | 52.14 | 35 | 37.14 | 47.14 | 80 | 17.14 | 46.67 | 46.67 | 55.07 | 33.97 | — | — | |
| Seed-1.6Setting=Single2026.02 | 70 | 18.57 | 64.29 | 15 | 62.86 | 25.71 | 58.57 | 32.86 | 46.67 | 33.33 | 63.29 | 21.92 | — | — | |
| Gemini-3-FlashSetting=Single2026.02 | 71.43 | 24.29 | 65.71 | 15.71 | 48.57 | 41.43 | 78.57 | 14.29 | 53.33 | 33.33 | 65.48 | 22.74 | — | — | |
| Seed-1.6Setting=Multi2026.02 | 78.57 | 5.71 | 70.71 | 7.14 | 70 | 18.57 | 78.57 | 11.43 | 60 | 0 | 73.15 | 9.59 | 15.6 | -56.3 | |
| Deepseek-v3.2Setting=Single2026.02 | 80 | 4.29 | 80 | 1.43 | 50 | 27.14 | 91.43 | 2.86 | 73.33 | 0 | 76.16 | 7.12 | — | — | |
| Claude-4.5-SonnetSetting=Multi2026.02 | 82.86 | 7.14 | 65.71 | 12.14 | 67.14 | 30 | 87.14 | 5.71 | 26.67 | 60 | 71.78 | 15.43 | 60.7 | -64.6 | |
| Gemini-3-FlashSetting=Multi2026.02 | 82.86 | 7.14 | 72.86 | 5.71 | 85.71 | 4.29 | 80 | 8.57 | 80 | 0 | 78.9 | 6.03 | 20.5 | -73.5 | |
| Qwen3-CoderSetting=Multi2026.02 | 88.57 | 1.43 | 67.86 | 13.57 | 80 | 10 | 90 | 1.43 | 60 | 0 | 78.08 | 7.67 | 41.8 | -77.4 | |
| Deepseek-v3.2Setting=Multi2026.02 | 88.57 | 0 | 80 | 0 | 85.71 | 4.29 | 92.86 | 1.43 | 73.33 | 0 | 84.93 | 1.1 | 11.5 | -84.5 |