Targeted Attack Detection on Alpaca OnlyTarget Short
100TPRPPL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PPLModel=Qwen2.5 14B, Attacks=Ignore2026.03 | 100 | 0.3008 | |
| PPLModel=Qwen2.5 14B, Attacks=Tree2026.03 | 100 | 0.3008 | |
| CleanGenModel=Qwen2.5 7B, Attacks=Word2026.03 | 100 | 0.776 | |
| CleanGenModel=Qwen2.5 7B, Attacks=Phrase2026.03 | 100 | 0.74 | |
| CleanGenModel=Qwen2.5 7B, Attacks=Sentence2026.03 | 100 | 0.768 | |
| STRIPModel=Qwen2.5 7B, Attacks=Joint2026.03 | 100 | 0.796 | |
| CleanGenModel=Qwen2.5 7B, Attacks=Joint2026.03 | 100 | 0.768 | |
| DualSentinelModel=Qwen2.5 14B, Attacks=Complete2026.03 | 99.6 | 0.03 | |
| PPLModel=Qwen2.5 7B, Attacks=Joint2026.03 | 99.19 | 0.344 | |
| PPLModel=Qwen2.5 7B, Attacks=Phrase2026.03 | 99.17 | 0.344 | |
| DualSentinelModel=Qwen2.5 7B, Attacks=Word2026.03 | 98.79 | 0.0564 | |
| STRIPModel=Qwen2.5 7B, Attacks=Word2026.03 | 98.39 | 0.748 | |
| PPLModel=Qwen2.5 14B, Attacks=Complete2026.03 | 98.38 | 0.3008 | |
| DualSentinelModel=Qwen2.5 7B, Attacks=Joint2026.03 | 98.19 | 0.057 | |
| STRIPModel=Qwen2.5 14B, Attacks=Complete2026.03 | 96.08 | 0.624 | |
| STRIPModel=Qwen2.5 14B, Attacks=Tree2026.03 | 95.23 | 0.624 | |
| STRIPModel=Qwen2.5 7B, Attacks=Sentence2026.03 | 94.89 | 0.804 | |
| DualSentinelModel=Qwen2.5 14B, Attacks=System2026.03 | 92.6 | 0.03 | |
| STRIPModel=Qwen2.5 7B, Attacks=Phrase2026.03 | 92.56 | 0.812 | |
| PPLModel=Qwen2.5 7B, Attacks=Word2026.03 | 92.36 | 0.344 | |
| DualSentinelModel=Qwen2.5 14B, Attacks=Ignore2026.03 | 91 | 0.03 | |
| DualSentinelModel=Qwen2.5 7B, Attacks=Sentence2026.03 | 89.61 | 0.062 | |
| DualSentinelModel=Qwen2.5 14B, Attacks=Tree2026.03 | 88.6 | 0.03 | |
| CleanGenModel=Qwen2.5 14B, Attacks=System2026.03 | 87.67 | 0.7355 | |
| STRIPModel=Qwen2.5 14B, Attacks=Ignore2026.03 | 85.6 | 0.624 | |
| DualSentinelModel=Qwen2.5 7B, Attacks=Phrase2026.03 | 79.85 | 0.058 | |
| PPLModel=Qwen2.5 14B, Attacks=System2026.03 | 77.73 | 0.3008 | |
| PPLModel=Qwen2.5 7B, Attacks=Sentence2026.03 | 76.59 | 0.344 | |
| STRIPModel=Qwen2.5 14B, Attacks=System2026.03 | 75.55 | 0.624 | |
| ONIONModel=Qwen2.5 7B, Attacks=Word2026.03 | 65.86 | 0.008 | |
| CleanGenModel=Qwen2.5 14B, Attacks=Ignore2026.03 | 63.92 | 0.7355 | |
| CleanGenModel=Qwen2.5 14B, Attacks=Tree2026.03 | 61.9 | 0.7355 | |
| ONIONModel=Qwen2.5 7B, Attacks=Joint2026.03 | 51.4 | 0.008 | |
| ParagraphModel=Qwen2.5 7B, Attacks=Word2026.03 | 48.19 | 0.504 | |
| ParagraphModel=Qwen2.5 7B, Attacks=Sentence2026.03 | 37.44 | 0.492 | |
| ParagraphModel=Qwen2.5 7B, Attacks=Phrase2026.03 | 32.64 | 0.4 | |
| CleanGenModel=Qwen2.5 14B, Attacks=Complete2026.03 | 13.76 | 0.7355 | |
| ParagraphModel=Qwen2.5 14B, Attacks=Complete2026.03 | 7.7 | 0.2896 | |
| ParagraphModel=Qwen2.5 14B, Attacks=Ignore2026.03 | 6.86 | 0.2896 | |
| ParagraphModel=Qwen2.5 14B, Attacks=System2026.03 | 4.9 | 0.2896 | |
| ParagraphModel=Qwen2.5 7B, Attacks=Joint2026.03 | 4.01 | 0.408 | |
| ONIONModel=Qwen2.5 7B, Attacks=Phrase2026.03 | 3.71 | 0.008 | |
| ParagraphModel=Qwen2.5 14B, Attacks=Tree2026.03 | 3.17 | 0.2896 | |
| ONIONModel=Qwen2.5 14B, Attacks=Complete2026.03 | 1.7 | 0.01 | |
| ONIONModel=Qwen2.5 7B, Attacks=Sentence2026.03 | 0.85 | 0.008 | |
| ONIONModel=Qwen2.5 14B, Attacks=Ignore2026.03 | 0.2 | 0.01 | |
| ConfGuardModel=Qwen2.5 14B, Attacks=Ignore2026.03 | 0.2 | 0.15 | |
| ConfGuardModel=Qwen2.5 14B, Attacks=Tree2026.03 | 0.2 | 0.15 | |
| ONIONModel=Qwen2.5 14B, Attacks=System2026.03 | 0.1 | 0.01 | |
| ConfGuardModel=Qwen2.5 14B, Attacks=Complete2026.03 | 0 | 0.15 | |
| ONIONModel=Qwen2.5 14B, Attacks=Tree2026.03 | 0 | 0.01 | |
| ConfGuardModel=Qwen2.5 14B, Attacks=System2026.03 | 0 | 0.15 | |
| ConfGuardModel=Qwen2.5 7B, Attacks=Word2026.03 | 0 | 0.1781 | |
| ConfGuardModel=Qwen2.5 7B, Attacks=Phrase2026.03 | 0 | 0.2 | |
| ConfGuardModel=Qwen2.5 7B, Attacks=Sentence2026.03 | 0 | 0.197 | |
| ConfGuardModel=Qwen2.5 7B, Attacks=Joint2026.03 | 0 | 0.187 |