Malicious Prompt Detection on VLGuard & MLLMGuard π = 0.005
94.37AUROCVLMGuard
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VLMGuardBackbone=Qwen, Type=Activation, Single inference=true, Single LM=true2024.10 | 94.37 | 94.79 | |
| VLMGuardBackbone=LLaVA, Type=Activation, Single inference=true, Single LM=true2024.10 | 91.94 | 93 | |
| HiddenDetectBackbone=LLaVA, Type=Activation, Single inference=true, Single LM=true2024.10 | 88.87 | 88.53 | |
| GPT-5.4Backbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.10 | 86.41 | 89.4 | |
| ASTRABackbone=LLaVA, Type=Activation, Single inference=true, Single LM=true2024.10 | 82.11 | 83.86 | |
| Qwen2.5-VL-72B-InstructBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.10 | 81.27 | 85.33 | |
| ASTRABackbone=Qwen, Type=Activation, Single inference=true, Single LM=true2024.10 | 80.52 | 82.7 | |
| InternVL3-78B-InstructBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.10 | 78.29 | 82.51 | |
| GradSafeBackbone=Qwen, Type=Uncertainty, Single inference=true, Single LM=true2024.10 | 76.87 | 78.32 | |
| GradSafeBackbone=LLaVA, Type=Uncertainty, Single inference=true, Single LM=true2024.10 | 76.12 | 77.1 | |
| Ovis2-34BBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.10 | 73.78 | 71.4 | |
| Gradient CuffBackbone=Qwen, Type=Uncertainty, Single inference=false, Single LM=true2024.10 | 73.57 | 80.56 | |
| Gradient CuffBackbone=LLaVA, Type=Uncertainty, Single inference=false, Single LM=true2024.10 | 71.77 | 78.55 | |
| HiddenDetectBackbone=Qwen, Type=Activation, Single inference=true, Single LM=true2024.10 | 70.34 | 76.57 | |
| Self-detectionBackbone=Qwen, Type=LLM, Single inference=true, Single LM=true2024.10 | 65.07 | 72.52 | |
| Self-detectionBackbone=LLaVA, Type=LLM, Single inference=true, Single LM=true2024.10 | 63.77 | 71.14 | |
| LLaVAGuardBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.10 | 62.75 | 59.62 | |
| LlamaGuard3-VisionBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.10 | 61.41 | 69.57 | |
| JailGuardBackbone=LLaVA, Type=Mutation, Single inference=false, Single LM=true2024.10 | 61.08 | 61.34 | |
| CIDERBackbone=LLaVA, Type=Denoising, Single inference=true, Single LM=false2024.10 | 61.05 | 59.99 | |
| JailGuardBackbone=Qwen, Type=Mutation, Single inference=false, Single LM=true2024.10 | 60.51 | 62.15 | |
| CIDERBackbone=Qwen, Type=Denoising, Single inference=true, Single LM=false2024.10 | 59.27 | 59.44 | |
| PerplexityBackbone=Qwen, Type=Uncertainty, Single inference=true, Single LM=true2024.10 | 54.19 | 63.68 | |
| MirrorCheckBackbone=Qwen, Type=Denoising, Single inference=true, Single LM=false2024.10 | 52.68 | 57.25 | |
| PerplexityBackbone=LLaVA, Type=Uncertainty, Single inference=true, Single LM=true2024.10 | 52.31 | 61.4 | |
| MirrorCheckBackbone=LLaVA, Type=Denoising, Single inference=true, Single LM=false2024.10 | 51.31 | 54.8 |