List your company

Agent Safety Evaluation on AgentDojo held-out

46.8ASRFATE
45.42454.7126473.288May 12, 2026

Evaluation results

MethodLinks
FATEBackbone=Gemma-3-12B-it2026.0546.846.29.1
FATEBackbone=Ministral-3-8B-Instruct2026.0548.643.87.4
FATEBackbone=Phi-4-reasoning2026.0550.342.98.9
FATEBackbone=Llama-3.1-8B-Instruct2026.0551.241.78.7
FATEBackbone=Qwen3-8B-Instruct2026.055439.28.2
BaseBackbone=Gemma-3-12B-it2026.0570.420.413.2
BaseBackbone=Ministral-3-8B-Instruct2026.0573.617.69.6
BaseBackbone=Phi-4-reasoning2026.0574.816.812.6
BaseBackbone=Llama-3.1-8B-Instruct2026.0576.815.811.8
BaseBackbone=Qwen3-8B-Instruct2026.0581.213.210.4