List your company
ResearchFormerly WizWand

Safety Guardrailing on HumanEval

0False Positive RatePerplexity Filter
-4235077Apr 1, 2026
Updated 7mo ago

Evaluation results

MethodLinks
Perplexity FilterBase Model=Llama-3-8B-Instruct2026.040
GradSafeBase Model=Llama-3-8B-Instruct2026.040
Llama GuardBase Model=Llama-3-8B-Instruct, Guardrail Stage=Pre-checking2026.040
Llama GuardBase Model=Llama-3-8B-Instruct, Guardrail Stage=Post-checking2026.040
SelfDefendBase Model=Llama-3-8B-Instruct, Defense Mode=Intent2026.040
WildGuardBase Model=Llama-3-8B-Instruct, Guardrail Stage=Pre-checking2026.040
WildGuardBase Model=Llama-3-8B-Instruct, Guardrail Stage=Post-checking2026.040
GuardReasonerBase Model=Llama-3-8B-Instruct, Guardrail Stage=Pre-checking2026.040
GuardReasonerBase Model=Llama-3-8B-Instruct, Guardrail Stage=Post-checking2026.040
SelfGraderBase Model=Llama-3-8B-Instruct2026.040
Perplexity FilterBackbone Model=Vicuna-13B-v1.5, Number of NTs (Q)=102026.040
GradientCuffBackbone Model=Vicuna-13B-v1.5, Number of NTs (Q)=102026.040
Llama GuardBackbone Model=Vicuna-13B-v1.5, Guardrail Processing Stage=Pre, Number of NTs (Q)=102026.040
Llama GuardBackbone Model=Vicuna-13B-v1.5, Guardrail Processing Stage=Post, Number of NTs (Q)=102026.040
SelfDefendBackbone Model=Vicuna-13B-v1.5, Guardrail Processing Stage=Intent, Number of NTs (Q)=102026.040
SelfGraderBackbone Model=Vicuna-13B-v1.5, Number of NTs (Q)=102026.040
Perplexity FilterBackbone=Qwen3.5-9B, Average=0.152026.040
GradientCuffBackbone=Qwen3.5-9B, Average=0.002026.040
Llama Guard (Pre)Backbone=Qwen3.5-9B, Average=1.452026.040
Llama Guard (Post)Backbone=Qwen3.5-9B, Average=0.432026.040
SelfDefend (Intent)Backbone=Qwen3.5-9B, Average=2.332026.040
WildGuard (Pre)Backbone=Qwen3.5-9B, Average=3.602026.040
WildGuard (Post)Backbone=Qwen3.5-9B, Average=0.182026.040
SelfGraderBackbone=Qwen3.5-9B, Average=0.002026.040
Prompt GuardBase Model=Llama-3-8B-Instruct2026.040.6
Prompt GuardBackbone Model=Vicuna-13B-v1.5, Number of NTs (Q)=102026.040.6
Prompt GuardBackbone=Qwen3.5-9B, Average=0.982026.040.6
SelfDefendBase Model=Llama-3-8B-Instruct, Defense Mode=Direct2026.041.21
SelfDefendBackbone Model=Vicuna-13B-v1.5, Guardrail Processing Stage=Direct, Number of NTs (Q)=102026.041.21
SelfDefend (Direct)Backbone=Qwen3.5-9B, Average=7.332026.041.8
GradientCuffBase Model=Llama-3-8B-Instruct2026.0451.21
Token HighlighterBase Model=Llama-3-8B-Instruct2026.04100