Model Accuracy on Model Utility Evaluation Set
30.22Accuracy (ACC)ChatBug
Evaluation Results
| Method | Links | |
|---|---|---|
| ChatBugModel=Llama3-8B, Attack Variant=Mismatch-∅2026.04 | 30.22 | |
| ChatBugModel=Qwen2.5-7B, Attack Variant=Mismatch-V2026.04 | 27.92 | |
| ChatBugModel=Llama3-8B, Attack Variant=Mismatch-V2026.04 | 27.85 | |
| ChatBugModel=Qwen2.5-7B, Attack Variant=Mismatch-∅2026.04 | 22.65 | |
| ChatBugModel=Llama2-7B, Attack Variant=Mismatch-C2026.04 | 17.2 | |
| ChatBugModel=DeepSeek-7B, Attack Variant=Mismatch-C2026.04 | 13.04 | |
| ChatBugModel=Llama3-8B, Attack Variant=Mismatch-C2026.04 | 11.33 | |
| ChatBugModel=Gemma3-4B, Attack Variant=Mismatch-V2026.04 | 9.1 | |
| ChatBugModel=Gemma3-4B, Attack Variant=Mismatch-∅2026.04 | 6.99 | |
| ChatBugModel=DeepSeek-7B, Attack Variant=Mismatch-∅2026.04 | 6.62 | |
| ChatBugModel=Gemma3-4B, Attack Variant=Mismatch-C2026.04 | 6.53 | |
| ChatBugModel=DeepSeek-7B, Attack Variant=Mismatch-V2026.04 | 3.72 | |
| ChatBugModel=Qwen3-8B, Attack Variant=Mismatch-∅2026.04 | 3.58 | |
| ChatBugModel=Qwen2.5-7B, Attack Variant=Mismatch-C2026.04 | 3.28 | |
| ChatBugModel=Qwen3-8B, Attack Variant=Mismatch-V2026.04 | 3.01 | |
| TemplateFuzzModel=Llama3-8B, Attack Variant=Top-1 Template2026.04 | 2.75 | |
| ChatBugModel=Qwen3-8B, Attack Variant=Mismatch-C2026.04 | 1.99 | |
| ChatBugModel=Llama2-7B, Attack Variant=Mismatch-V2026.04 | 1.71 | |
| TemplateFuzzModel=Qwen3-8B, Attack Variant=Top-1 Template2026.04 | 1.45 | |
| TemplateFuzzModel=DeepSeek-7B, Attack Variant=Top-1 Template2026.04 | 1.41 | |
| ChatBugModel=Llama2-7B, Attack Variant=Mismatch-∅2026.04 | 1.18 | |
| TemplateFuzzModel=Gemma3-4B, Attack Variant=Top-1 Template2026.04 | 1.01 | |
| TemplateFuzzModel=Llama2-7B, Attack Variant=Top-1 Template2026.04 | 0.21 | |
| TemplateFuzzModel=Qwen2.5-7B, Attack Variant=Top-1 Template2026.04 | 0.04 |