Adversarial and Jailbreaking Attack Detection on XSTest
0.984AUROCGradSafe
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GradSafeTarget Model=Llama-3.1-8B-Instruct, Grad=true2026.05 | 0.984 | — | |
| Linear ProbeTarget Model=Mistral-7B-Instruct-v0.3, Params=<1M, Grad=false2026.05 | 0.97 | — | |
| GradSafeTarget Model=Mistral-7B-Instruct-v0.3, Grad=true2026.05 | 0.95 | — | |
| Linear ProbeTarget Model=Llama-3.1-8B-Instruct, Params=<1M, Grad=false2026.05 | 0.949 | — | |
| SALOTarget Model=Llama-3.1-8B-Instruct, Params=<20M, Grad=false2026.05 | 0.948 | — | |
| SALOTarget Model=Qwen2.5-7B-Instruct, Params=<20M, Grad=false2026.05 | 0.946 | — | |
| GradSafeTarget Model=Qwen2.5-7B-Instruct, Grad=true2026.05 | 0.942 | — | |
| SALOTarget Model=Mistral-7B-Instruct-v0.3, Params=<20M, Grad=false2026.05 | 0.932 | — | |
| Linear ProbeTarget Model=Qwen2.5-7B-Instruct, Params=<1M, Grad=false2026.05 | 0.93 | — | |
| Smooth LLMTarget Model=Qwen2.5-7B-Instruct, Grad=false2026.05 | 0.893 | — | |
| DUOGUARDMode=Zero-shot2026.02 | 0.8418 | 0.781 | |
| OpenAI OmniMode=Zero-shot2026.02 | 0.8257 | 0.9667 | |
| LLAMAGUARD4-12BMode=Zero-shot2026.02 | 0.812 | 1 | |
| FDBDMode=Zero-shot2026.02 | 0.8021 | 0.781 | |
| POLYGUARDMode=Zero-shot2026.02 | 0.8007 | 0.8714 | |
| Perspective APIMode=Zero-shot2026.02 | 0.7932 | 0.7381 | |
| WILDGUARDMode=Zero-shot2026.02 | 0.7929 | 1 | |
| MDJUDGEMode=Zero-shot2026.02 | 0.7906 | 0.9238 | |
| Smooth LLMTarget Model=Mistral-7B-Instruct-v0.3, Grad=false2026.05 | 0.786 | — | |
| LLAMAGUARD3-1BMode=Zero-shot2026.02 | 0.7843 | 1 | |
| Smooth LLMTarget Model=Llama-3.1-8B-Instruct, Grad=false2026.05 | 0.716 | — | |
| T3-GMMMode=Zero-shot2026.02 | 0.6794 | 0.8143 | |
| LLAMAGUARD3-1B-LOGITSMode=Zero-shot2026.02 | 0.6542 | 0.9095 | |
| OPENMAXMode=Zero-shot2026.02 | 0.6308 | 0.9952 | |
| PPL FilterTarget Model=Qwen2.5-7B-Instruct, Grad=false2026.05 | 0.595 | — | |
| T3-OCSVMMode=Zero-shot2026.02 | 0.58 | 0.8762 | |
| PPL FilterTarget Model=Mistral-7B-Instruct-v0.3, Grad=false2026.05 | 0.57 | — | |
| PPL FilterTarget Model=Llama-3.1-8B-Instruct, Grad=false2026.05 | 0.481 | — | |
| RMDMode=Zero-shot2026.02 | 0.373 | 1 | |
| CIDERMode=Zero-shot2026.02 | 0.3345 | 1 | |
| ADASCALEMode=Zero-shot2026.02 | 0.2803 | 0.9952 | |
| REACTMode=Zero-shot2026.02 | 0.2373 | 1 | |
| GMMMode=Zero-shot2026.02 | 0.2298 | 1 | |
| NNGUIDEMode=Zero-shot2026.02 | 0.2144 | 1 | |
| VIMMode=Zero-shot2026.02 | 0.1565 | 1 |