Cascade Attack Detection on AutoGen
86.8TPR@5%CASPIAN
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CASPIAN2026.05 | 86.8 | 79.2 | 0.942 | |
| Full TraceCategory=LLM Judges, Evaluation Mode=offline2026.05 | 71.3 | — | 0.811 | |
| BlindGuardCategory=Graph-Based2026.05 | 64.5 | 52.1 | 0.815 | |
| GUARDIANCategory=Graph-Based2026.05 | 63.2 | 49.8 | 0.818 | |
| G-SafeguardCategory=Graph-Based2026.05 | 61.6 | 48.2 | 0.802 | |
| Sliding WindowCategory=LLM Judges, Window size (W)=102026.05 | 61.2 | 44.1 | 0.763 | |
| Sliding WindowCategory=LLM Judges, Window size (W)=52026.05 | 52.8 | 33.6 | 0.742 | |
| Single TurnCategory=LLM Judges2026.05 | 41.2 | 21.5 | 0.694 | |
| PromptArmorCategory=Guardrails2026.05 | 36.1 | 25.1 | 0.664 | |
| JailGuardCategory=Guardrails2026.05 | 32.6 | 24.2 | 0.63 | |
| Prompt Guard 2Category=Guardrails2026.05 | 28.4 | 19.8 | 0.618 | |
| PerplexityCategory=Distributional2026.05 | 13.2 | 7.6 | 0.578 |