Malicious Prompt Detection on VLGuard & MSSBench π = 0.005
0.9195AUROC (VLGuard & MSSBench)VLMGuard
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| VLMGuardBackbone=Qwen, Type=Activation, Single inference=true, Single LM=true2024.10 | 0.9195 | 0.9479 | |
| VLMGuardBackbone=LLaVA, Type=Activation, Single inference=true, Single LM=true2024.10 | 0.9047 | 0.93 | |
| GPT-5.4Backbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.10 | 0.8762 | 0.894 | |
| HiddenDetectBackbone=LLaVA, Type=Activation, Single inference=true, Single LM=true2024.10 | 0.8646 | 0.8853 | |
| Qwen2.5-VL-72B-InstructBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.10 | 0.8295 | 0.8533 | |
| InternVL3-78B-InstructBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.10 | 0.8003 | 0.8251 | |
| ASTRABackbone=LLaVA, Type=Activation, Single inference=true, Single LM=true2024.10 | 0.7934 | 0.8386 | |
| GradSafeBackbone=Qwen, Type=Uncertainty, Single inference=true, Single LM=true2024.10 | 0.7906 | 0.7832 | |
| GradSafeBackbone=LLaVA, Type=Uncertainty, Single inference=true, Single LM=true2024.10 | 0.7782 | 0.771 | |
| HiddenDetectBackbone=Qwen, Type=Activation, Single inference=true, Single LM=true2024.10 | 0.7604 | 0.7657 | |
| Ovis2-34BBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.10 | 0.7558 | 0.714 | |
| Gradient CuffBackbone=Qwen, Type=Uncertainty, Single inference=false, Single LM=true2024.10 | 0.7355 | 0.8056 | |
| ASTRABackbone=Qwen, Type=Activation, Single inference=true, Single LM=true2024.10 | 0.7354 | 0.827 | |
| Gradient CuffBackbone=LLaVA, Type=Uncertainty, Single inference=false, Single LM=true2024.10 | 0.7124 | 0.7855 | |
| Self-detectionBackbone=Qwen, Type=LLM, Single inference=true, Single LM=true2024.10 | 0.6633 | 0.7252 | |
| Self-detectionBackbone=LLaVA, Type=LLM, Single inference=true, Single LM=true2024.10 | 0.6512 | 0.7114 | |
| LLaVAGuardBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.10 | 0.6446 | 0.5962 | |
| LlamaGuard3-VisionBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.10 | 0.619 | 0.6957 | |
| JailGuardBackbone=Qwen, Type=Mutation, Single inference=false, Single LM=true2024.10 | 0.617 | 0.6215 | |
| CIDERBackbone=LLaVA, Type=Denoising, Single inference=true, Single LM=false2024.10 | 0.6157 | 0.5999 | |
| JailGuardBackbone=LLaVA, Type=Mutation, Single inference=false, Single LM=true2024.10 | 0.6022 | 0.6134 | |
| CIDERBackbone=Qwen, Type=Denoising, Single inference=true, Single LM=false2024.10 | 0.5964 | 0.5944 | |
| MirrorCheckBackbone=Qwen, Type=Denoising, Single inference=true, Single LM=false2024.10 | 0.5268 | 0.5725 | |
| PerplexityBackbone=Qwen, Type=Uncertainty, Single inference=true, Single LM=true2024.10 | 0.5201 | 0.6368 | |
| PerplexityBackbone=LLaVA, Type=Uncertainty, Single inference=true, Single LM=true2024.10 | 0.5071 | 0.614 | |
| MirrorCheckBackbone=LLaVA, Type=Denoising, Single inference=true, Single LM=false2024.10 | 0.5017 | 0.548 |