Reading Comprehension on SQuAD (Usage/Attack Metrics)
75.91Attack AccuracyNo-shield
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| No-shieldBackbone=LLaMA3-8B2024.10 | 75.91 | — | — | |
| NPLOBackbone=LLaMA3-8B2024.10 | 75.2 | — | — | |
| DarkneTZBackbone=LLaMA3-8B2024.10 | 73.81 | — | — | |
| SOTERBackbone=LLaMA3-8B2024.10 | 69.12 | — | — | |
| NPLOBackbone=ChatGLM3-6B2024.10 | 68.21 | — | — | |
| No-shieldBackbone=ChatGLM3-6B2024.10 | 68.13 | — | — | |
| SerdabBackbone=LLaMA3-8B2024.10 | 67.92 | — | — | |
| SerdabBackbone=ChatGLM3-6B2024.10 | 66.28 | — | — | |
| No-shieldBackbone=Gemma2-2B2024.10 | 63.96 | — | — | |
| NPLOBackbone=Gemma2-2B2024.10 | 63.94 | — | — | |
| DarkneTZBackbone=ChatGLM3-6B2024.10 | 63.91 | — | — | |
| SOTERBackbone=ChatGLM3-6B2024.10 | 62.61 | — | — | |
| DarkneTZBackbone=Gemma2-2B2024.10 | 61.02 | — | — | |
| SerdabBackbone=Gemma2-2B2024.10 | 60.82 | — | — | |
| SOTERBackbone=Gemma2-2B2024.10 | 57.87 | — | — | |
| NPLOBackbone=Qwen2-0.5B2024.10 | 32.33 | — | — | |
| No-shieldBackbone=Qwen2-0.5B2024.10 | 30.54 | — | — | |
| DarkneTZBackbone=Qwen2-0.5B2024.10 | 29.89 | — | — | |
| SerdabBackbone=Qwen2-0.5B2024.10 | 28.42 | — | — | |
| SOTERBackbone=Qwen2-0.5B2024.10 | 26.34 | — | — | |
| TLGBackbone=LLaMA3-8B2024.10 | 11.94 | — | — | |
| DTEBackbone=LLaMA3-8B2024.10 | 10.48 | — | — | |
| CoreGuardBackbone=LLaMA3-8B2024.10 | 10.01 | 0 | 73.02 | |
| Black-boxBackbone=LLaMA3-8B2024.10 | 9.71 | — | — | |
| ShadowNetBackbone=LLaMA3-8B2024.10 | 9.64 | — | — | |
| ShadowNetBackbone=ChatGLM3-6B2024.10 | 9.56 | — | — | |
| DTEBackbone=ChatGLM3-6B2024.10 | 9.42 | — | — | |
| Black-boxBackbone=ChatGLM3-6B2024.10 | 9.15 | — | — | |
| CoreGuardBackbone=ChatGLM3-6B2024.10 | 8.98 | 0 | 62.11 | |
| Black-boxBackbone=Gemma2-2B2024.10 | 8.81 | — | — | |
| TLGBackbone=ChatGLM3-6B2024.10 | 8.61 | — | — | |
| TLGBackbone=Gemma2-2B2024.10 | 7.91 | — | — | |
| CoreGuardBackbone=Gemma2-2B2024.10 | 7.81 | 0 | 43.21 | |
| DTEBackbone=Gemma2-2B2024.10 | 7.51 | — | — | |
| CoreGuardBackbone=Qwen2-0.5B2024.10 | 7.35 | 0 | 16.5 | |
| TLGBackbone=Qwen2-0.5B2024.10 | 6.81 | — | — | |
| ShadowNetBackbone=Gemma2-2B2024.10 | 6.71 | — | — | |
| ShadowNetBackbone=Qwen2-0.5B2024.10 | 5.93 | — | — | |
| Black-boxBackbone=Qwen2-0.5B2024.10 | 5.66 | — | — | |
| DTEBackbone=Qwen2-0.5B2024.10 | 4.42 | — | — |