Harmful behavior detection on Constructed in-domain dataset (test)
99.2AccuracyWebAgentGuard-8B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| WebAgentGuard-8BType=Ours, Parameters=8B2026.04 | 99.2 | 98.4 | 100 | 99.19 | |
| WebAgentGuard-4BType=Ours, Parameters=4B2026.04 | 98.2 | 96.8 | 99.59 | 98.17 | |
| GPT-4.1Type=Closed-source API2026.04 | 74.9 | 49.8 | 100 | 66.48 | |
| GPT-4oType=Closed-source API2026.04 | 70 | 40.2 | 100 | 57.34 | |
| GPT-4o-MiniType=Closed-source API2026.04 | 62.5 | 32 | 82.05 | 46.04 | |
| Qwen3-VL-Instruct-4BType=Open-source instructed model, Parameters=4B2026.04 | 58.2 | 17.33 | 100 | 29.55 | |
| Prompt-Guard-1-86MType=Guard model, Parameters=86M2026.04 | 56.2 | 93.4 | 53.56 | 68.08 | |
| Qwen3-VL-Instruct-8BType=Open-source instructed model, Parameters=8B2026.04 | 53.2 | 6.4 | 100 | 12.03 | |
| Llama-3.2-Vision-Instruct-11BType=Open-source instructed model, Parameters=11B2026.04 | 51.9 | 10.08 | 73.13 | 17.72 | |
| Qwen2.5-VL-Instruct-7BType=Open-source instructed model, Parameters=7B2026.04 | 50.9 | 3.51 | 73.91 | 6.7 | |
| Prompt-Guard-2-86MType=Guard model, Parameters=86M2026.04 | 50.7 | 2.6 | 68.42 | 5.01 | |
| GuardReasoner-VL-7BType=Guard model, Parameters=7B2026.04 | 50 | 0.2 | 50 | 3.98 | |
| Llama-Guard3-Vision-11BType=Guard model, Parameters=11B2026.04 | 49.6 | 1.4 | 38.89 | 2.7 |