Guardrail False Positive Rate Estimation on GSM8K benign prompts
0False Positive RatePerplexity Filter
Evaluation results
| Method | Links | |
|---|---|---|
| Perplexity FilterBackbone Model=Vicuna-13B-v1.5, Number of NTs (Q)=102026.04 | 0 | |
| Prompt GuardBackbone Model=Vicuna-13B-v1.5, Number of NTs (Q)=102026.04 | 0 | |
| Llama GuardBackbone Model=Vicuna-13B-v1.5, Guardrail Processing Stage=Pre, Number of NTs (Q)=102026.04 | 0 | |
| Llama GuardBackbone Model=Vicuna-13B-v1.5, Guardrail Processing Stage=Post, Number of NTs (Q)=102026.04 | 0 | |
| SelfDefendBackbone Model=Vicuna-13B-v1.5, Guardrail Processing Stage=Intent, Number of NTs (Q)=102026.04 | 0 | |
| SelfGraderBackbone Model=Vicuna-13B-v1.5, Number of NTs (Q)=102026.04 | 0 | |
| SelfDefendBackbone Model=Vicuna-13B-v1.5, Guardrail Processing Stage=Direct, Number of NTs (Q)=102026.04 | 0.66 | |
| GradientCuffBackbone Model=Vicuna-13B-v1.5, Number of NTs (Q)=102026.04 | 18 |