Mathematical Reasoning on GSM8k (Usage/Attack Accuracy)
55.95Attack AccuracyNo-shield
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| No-shieldBackbone=ChatGLM3-6B2024.10 | 55.95 | — | — | |
| NPLOBackbone=ChatGLM3-6B2024.10 | 55.07 | — | — | |
| DarkneTZBackbone=ChatGLM3-6B2024.10 | 54.91 | — | — | |
| SOTERBackbone=ChatGLM3-6B2024.10 | 54.55 | — | — | |
| NPLOBackbone=LLaMA3-8B2024.10 | 53.83 | — | — | |
| SerdabBackbone=ChatGLM3-6B2024.10 | 53.67 | — | — | |
| No-shieldBackbone=LLaMA3-8B2024.10 | 53.07 | — | — | |
| DarkneTZBackbone=LLaMA3-8B2024.10 | 51.31 | — | — | |
| SOTERBackbone=LLaMA3-8B2024.10 | 49.75 | — | — | |
| SerdabBackbone=LLaMA3-8B2024.10 | 47.79 | — | — | |
| No-shieldBackbone=Gemma2-2B2024.10 | 40.94 | — | — | |
| NPLOBackbone=Gemma2-2B2024.10 | 40.5 | — | — | |
| DarkneTZBackbone=Gemma2-2B2024.10 | 37.07 | — | — | |
| SerdabBackbone=Gemma2-2B2024.10 | 35.18 | — | — | |
| SOTERBackbone=Gemma2-2B2024.10 | 32.67 | — | — | |
| No-shieldBackbone=Qwen2-0.5B2024.10 | 21.53 | — | — | |
| NPLOBackbone=Qwen2-0.5B2024.10 | 20.92 | — | — | |
| DarkneTZBackbone=Qwen2-0.5B2024.10 | 16.81 | — | — | |
| SerdabBackbone=Qwen2-0.5B2024.10 | 14.96 | — | — | |
| SOTERBackbone=Qwen2-0.5B2024.10 | 12.5 | — | — | |
| ShadowNetBackbone=Gemma2-2B2024.10 | 10.81 | — | — | |
| CoreGuardBackbone=LLaMA3-8B2024.10 | 6.22 | 0 | 33.11 | |
| DTEBackbone=LLaMA3-8B2024.10 | 6.09 | — | — | |
| TLGBackbone=LLaMA3-8B2024.10 | 5.61 | — | — | |
| TLGBackbone=Gemma2-2B2024.10 | 4.58 | — | — | |
| DTEBackbone=Gemma2-2B2024.10 | 4.56 | — | — | |
| ShadowNetBackbone=LLaMA3-8B2024.10 | 4.15 | — | — | |
| Black-boxBackbone=LLaMA3-8B2024.10 | 4.05 | — | — | |
| CoreGuardBackbone=Gemma2-2B2024.10 | 3.91 | 0 | 30.1 | |
| TLGBackbone=ChatGLM3-6B2024.10 | 2.84 | — | — | |
| CoreGuardBackbone=Qwen2-0.5B2024.10 | 2.41 | 0 | 15.51 | |
| DTEBackbone=Qwen2-0.5B2024.10 | 2.36 | — | — | |
| Black-boxBackbone=Gemma2-2B2024.10 | 1.74 | — | — | |
| TLGBackbone=Qwen2-0.5B2024.10 | 1.43 | — | — | |
| ShadowNetBackbone=Qwen2-0.5B2024.10 | 1.34 | — | — | |
| Black-boxBackbone=Qwen2-0.5B2024.10 | 1.29 | — | — | |
| CoreGuardBackbone=ChatGLM3-6B2024.10 | 1.04 | 0 | 37.13 | |
| DTEBackbone=ChatGLM3-6B2024.10 | 0.93 | — | — | |
| ShadowNetBackbone=ChatGLM3-6B2024.10 | 0.43 | — | — | |
| Black-boxBackbone=ChatGLM3-6B2024.10 | 0.23 | — | — |