List your company
ResearchFormerly WizWand

Malicious Prompt Detection on VLGuard & MSSBench π = 0.005

0.9195AUROC (VLGuard & MSSBench)VLMGuard
0.4849880.5977940.71060.823406Oct 1, 2024
Updated 4mo ago

Evaluation results

MethodLinks
VLMGuardBackbone=Qwen, Type=Activation, Single inference=true, Single LM=true2024.100.91950.9479
VLMGuardBackbone=LLaVA, Type=Activation, Single inference=true, Single LM=true2024.100.90470.93
GPT-5.4Backbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.100.87620.894
HiddenDetectBackbone=LLaVA, Type=Activation, Single inference=true, Single LM=true2024.100.86460.8853
Qwen2.5-VL-72B-InstructBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.100.82950.8533
InternVL3-78B-InstructBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.100.80030.8251
ASTRABackbone=LLaVA, Type=Activation, Single inference=true, Single LM=true2024.100.79340.8386
GradSafeBackbone=Qwen, Type=Uncertainty, Single inference=true, Single LM=true2024.100.79060.7832
GradSafeBackbone=LLaVA, Type=Uncertainty, Single inference=true, Single LM=true2024.100.77820.771
HiddenDetectBackbone=Qwen, Type=Activation, Single inference=true, Single LM=true2024.100.76040.7657
Ovis2-34BBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.100.75580.714
Gradient CuffBackbone=Qwen, Type=Uncertainty, Single inference=false, Single LM=true2024.100.73550.8056
ASTRABackbone=Qwen, Type=Activation, Single inference=true, Single LM=true2024.100.73540.827
Gradient CuffBackbone=LLaVA, Type=Uncertainty, Single inference=false, Single LM=true2024.100.71240.7855
Self-detectionBackbone=Qwen, Type=LLM, Single inference=true, Single LM=true2024.100.66330.7252
Self-detectionBackbone=LLaVA, Type=LLM, Single inference=true, Single LM=true2024.100.65120.7114
LLaVAGuardBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.100.64460.5962
LlamaGuard3-VisionBackbone=N/A, Type=LLM, Single inference=true, Single LM=false2024.100.6190.6957
JailGuardBackbone=Qwen, Type=Mutation, Single inference=false, Single LM=true2024.100.6170.6215
CIDERBackbone=LLaVA, Type=Denoising, Single inference=true, Single LM=false2024.100.61570.5999
JailGuardBackbone=LLaVA, Type=Mutation, Single inference=false, Single LM=true2024.100.60220.6134
CIDERBackbone=Qwen, Type=Denoising, Single inference=true, Single LM=false2024.100.59640.5944
MirrorCheckBackbone=Qwen, Type=Denoising, Single inference=true, Single LM=false2024.100.52680.5725
PerplexityBackbone=Qwen, Type=Uncertainty, Single inference=true, Single LM=true2024.100.52010.6368
PerplexityBackbone=LLaVA, Type=Uncertainty, Single inference=true, Single LM=true2024.100.50710.614
MirrorCheckBackbone=LLaVA, Type=Denoising, Single inference=true, Single LM=false2024.100.50170.548