Multi-turn attack detection on Combined LMSYS, SafeDialBench, Synthetic (held-out)
99Detection AccuracyCum. drift threshold
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Cum. drift thresholdType=Unsupervised (no training)2026.04 | 99 | 29 | — | |
| Lakera GuardType=API, Evaluation protocol=zero-shot2026.04 | 95.2 | 76.3 | 32.7 | |
| TF-IDF onlyDimensions=5,000 dims, Evaluation protocol=trained on in-distribution data2026.04 | 95.1 | 58.6 | — | |
| TF-IDF + text scalarsEvaluation protocol=trained on in-distribution data2026.04 | 93.8 | 46.3 | — | |
| LAD (Qwen 2.5 32B)Backbone=Qwen 2.5 32B, Evaluation protocol=trained on in-distribution data2026.04 | 89.4 | 2.4 | 2 | |
| LAD (Mistral 3.1 24B)Backbone=Mistral 3.1 24B, Evaluation protocol=trained on in-distribution data2026.04 | 87.7 | 2.8 | 2.3 | |
| LAD (Llama 3.1 70B)Backbone=Llama 3.1 70B, Evaluation protocol=trained on in-distribution data2026.04 | 87.3 | 2.9 | 2.1 | |
| LAD (Gemma 3 27B)Backbone=Gemma 3 27B, Evaluation protocol=trained on in-distribution data2026.04 | 85.3 | 4 | 2.1 | |
| LLM GuardBackbone=DeBERTa-v3, Evaluation protocol=zero-shot2026.04 | 29 | 27.9 | 6.8 | |
| PromptGuardModel size=86M, Evaluation protocol=zero-shot2026.04 | 19.8 | 16.1 | 3.5 |