List your company
ResearchFormerly WizWand

Jailbreak Defense on Manual (IJP)

0ASRPrompt Guard
-1.89610.90223.736.498Apr 1, 2026
Updated 7mo ago

Evaluation results

MethodLinks
Prompt GuardLatency (Sec.)=15.11, Memory Overhead (MB)=1064.732026.0400
Prompt GuardLatency (Sec.)=16.87, Memory Overhead (MB)=1064.712026.0400
Prompt GuardBackbone=Qwen2.5-7B-Instruct, Latency (Sec.)=8.48, Memory Overhead (MB)=1154.582026.0400
WildGuardBackbone=Qwen2.5-7B-Instruct, Latency (Sec.)=1.00, Memory Overhead (MB)=13939.25, Evaluation Stage=Pre2026.0403.4
GuardReasonerBackbone=Qwen2.5-7B-Instruct, Evaluation Stage=Pre2026.0400.9
SelfGraderBackbone=Qwen2.5-7B-Instruct, Latency (Sec.)=1.33, Memory Overhead (MB)=374.252026.0400
SelfGraderTarget LLM=LLama-3-8B Instruct2026.040.29.1
WildGuard (Pre)Latency (Sec.)=2.74, Memory Overhead (MB)=28014.702026.040.43.3
SelfGraderLatency (Sec.)=0.78, Memory Overhead (MB)=377.612026.040.56.3
GradientCuffLatency (Sec.)=19.26, Memory Overhead (MB)=1223.192026.040.83.6
GuardReasoner (Pre)Latency (Sec.)=11.90, Memory Overhead (MB)=15317.412026.040.97.8
SelfDefendvariant=Direct, Target LLM=LLama-3-8B Instruct2026.040.927
SelfDefendBackbone=Qwen2.5-7B-Instruct, Latency (Sec.)=0.64, Memory Overhead (MB)=13130.75, Mode=Direct2026.041.725.3
GuardReasonerBackbone=Qwen2.5-7B-Instruct, Evaluation Stage=Post2026.041.851.9
SelfDefend (Direct)Latency (Sec.)=0.80, Memory Overhead (MB)=13241.862026.04228.9
SelfDefend (Intent)Latency (Sec.)=2.38, Memory Overhead (MB)=13247.352026.04225.7
WildGuard (Post)Latency (Sec.)=2.78, Memory Overhead (MB)=28098.392026.04286.2
WildGuardBackbone=Qwen2.5-7B-Instruct, Latency (Sec.)=0.94, Memory Overhead (MB)=13995.25, Evaluation Stage=Post2026.04256.9
GuardReasoner (Post)Latency (Sec.)=13.14, Memory Overhead (MB)=15317.422026.042.286.2
SelfDefendBackbone=Qwen2.5-7B-Instruct, Latency (Sec.)=2.02, Memory Overhead (MB)=13137.25, Mode=Intent2026.042.429.1
Token HighlighterLatency (Sec.)=15.20, Memory Overhead (MB)=65536.662026.043.610
SelfDefendDefense Protocol=Direct, Latency (Sec.)=0.84, Memory Overhead (MB)=13183.912026.045.426
SelfDefendDefense Protocol=Intent, Latency (Sec.)=2.14, Memory Overhead (MB)=13194.112026.045.729.1
Llama Guard (Pre)Latency (Sec.)=0.62, Memory Overhead (MB)=13697.272026.04656.1
Llama Guard (Post)Latency (Sec.)=0.70, Memory Overhead (MB)=13783.602026.046.196.6
Llama Guardvariant=Pre, Target LLM=LLama-3-8B Instruct2026.046.462.4
LLama-3-8B-Instruct (No Defense)Latency (Sec.)=0.862026.047.8—
Perplexity FilterLatency (Sec.)=0.27, Memory Overhead (MB)=13279.462026.047.8100
Llama GuardBackbone=Qwen2.5-7B-Instruct, Latency (Sec.)=0.36, Memory Overhead (MB)=13592.75, Evaluation Stage=Pre2026.048.156.1
GradientCuffBackbone=Qwen2.5-7B-Instruct, Latency (Sec.)=80.70, Memory Overhead (MB)=1672.502026.0410.227.7
Llama GuardBackbone=Qwen2.5-7B-Instruct, Latency (Sec.)=0.45, Memory Overhead (MB)=13739.13, Evaluation Stage=Post2026.0410.579.2
Llama GuardDeployment Stage=Pre, Latency (Sec.)=0.49, Memory Overhead (MB)=13649.822026.0419.456.1
Llama GuardDeployment Stage=Post, Latency (Sec.)=0.56, Memory Overhead (MB)=13753.872026.042568.4
SelfGraderQ=10, Latency (Sec.)=0.74, Memory Overhead (MB)=2765.652026.0425.155.2
Qwen2.5-7B-Instruct (No Defense)Backbone=Qwen2.5-7B-Instruct, Latency (Sec.)=5.292026.0425.7—
Perplexity FilterBackbone=Qwen2.5-7B-Instruct, Latency (Sec.)=0.10, Memory Overhead (MB)=13134.882026.0425.7100
Vicuna-13B-v1.5Defense=No Defense, Latency (Sec.)=3.202026.0447.4—
Perplexity FilterLatency (Sec.)=0.17, Memory Overhead (MB)=13214.012026.0447.4100