Abnormal Behavior Detection on JailBreakV (test)
100AccuracyAbnorDetector-Full
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| AbnorDetector-FullBase Model=Llama-2-7b-chat, alpha (α)=0.25, beta (β)=0.252025.04 | 100 | 0 | 99.5 | |
| AbnorDetector-FullBase Model=Llama-2-7b-chat, alpha (α)=0.125, beta (β)=0.1252025.04 | 100 | 0 | 99.5 | |
| AbnorDetector-FullBase Model=Llama-3.1-70B-Instruct, alpha (α)=0.25, beta (β)=0.252025.04 | 100 | 0 | 100 | |
| AbnorDetector-FullBase Model=Llama-3.1-70B-Instruct, alpha (α)=0.125, beta (β)=0.1252025.04 | 100 | 0 | 100 | |
| AbnorDetector-FullBase Model=Llama-3.1-8B-Instruct, alpha (α)=0.25, beta (β)=0.252025.04 | 99.33 | 0.2222 | 99.33 | |
| AbnorDetector-LiteBase Model=Llama-2-7b-chat, alpha (α)=0.5, beta (β)=0.52025.04 | 98.67 | 0.2222 | 98.83 | |
| AbnorDetector-LiteBase Model=Llama-2-7b-chat, alpha (α)=0.25, beta (β)=0.252025.04 | 98.67 | 0.2222 | 99.17 | |
| AbnorDetector-LiteBase Model=Llama-3.1-70B-Instruct, alpha (α)=0.5, beta (β)=0.52025.04 | 98 | 0.6667 | 98 | |
| AbnorDetector-FullBase Model=Gemma-7b-it, alpha (α)=0.25, beta (β)=0.252025.04 | 96.67 | 0.2222 | 97.48 | |
| AbnorDetector-FullBase Model=Llama-3.1-8B-Instruct, alpha (α)=0.125, beta (β)=0.1252025.04 | 95.67 | 22.8889 | 97.62 | |
| AbnorDetector-LiteBase Model=Llama-3.1-70B-Instruct, alpha (α)=0.25, beta (β)=0.252025.04 | 95.33 | 0.2222 | 96.46 | |
| AbnorDetector-LiteBase Model=Gemma-7b-it, alpha (α)=0.25, beta (β)=0.252025.04 | 95 | 0 | 93.6 | |
| AbnorDetector-LiteBase Model=Gemma-7b-it, alpha (α)=0.5, beta (β)=0.52025.04 | 93.33 | 1.5556 | 94.43 | |
| AbnorDetector-FullBase Model=Gemma-7b-it, alpha (α)=0.125, beta (β)=0.1252025.04 | 92 | 12.6667 | 95.34 | |
| GradSafeBase Model=Llama-2-7b-chat2025.04 | 91 | — | — | |
| AbnorDetector-LiteBase Model=Llama-3.1-8B-Instruct, alpha (α)=0.5, beta (β)=0.52025.04 | 90.67 | 2.8889 | 94.12 | |
| AbnorDetector-LiteBase Model=Llama-3.1-8B-Instruct, alpha (α)=0.25, beta (β)=0.252025.04 | 89.33 | 1.5556 | 89.93 |