Text-to-SQL on Spider (Security/Usage Accuracy)
41.73Attack AccuracyNPLO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| NPLOBackbone=LLaMA3-8B2024.10 | 41.73 | — | — | |
| No-shieldBackbone=LLaMA3-8B2024.10 | 40.04 | — | — | |
| No-shieldBackbone=Gemma2-2B2024.10 | 39.15 | — | — | |
| NPLOBackbone=Gemma2-2B2024.10 | 38.83 | — | — | |
| SerdabBackbone=LLaMA3-8B2024.10 | 38.27 | — | — | |
| DarkneTZBackbone=LLaMA3-8B2024.10 | 38.14 | — | — | |
| NPLOBackbone=ChatGLM3-6B2024.10 | 37.03 | — | — | |
| SOTERBackbone=LLaMA3-8B2024.10 | 36.63 | — | — | |
| No-shieldBackbone=ChatGLM3-6B2024.10 | 35.81 | — | — | |
| DarkneTZBackbone=ChatGLM3-6B2024.10 | 33.81 | — | — | |
| SOTERBackbone=ChatGLM3-6B2024.10 | 33.22 | — | — | |
| SerdabBackbone=ChatGLM3-6B2024.10 | 32.25 | — | — | |
| NPLOBackbone=Qwen2-0.5B2024.10 | 30.28 | — | — | |
| No-shieldBackbone=Qwen2-0.5B2024.10 | 28.48 | — | — | |
| DarkneTZBackbone=Qwen2-0.5B2024.10 | 26.01 | — | — | |
| SerdabBackbone=Gemma2-2B2024.10 | 24.8 | — | — | |
| SerdabBackbone=Qwen2-0.5B2024.10 | 23.9 | — | — | |
| DarkneTZBackbone=Gemma2-2B2024.10 | 23.29 | — | — | |
| SOTERBackbone=Qwen2-0.5B2024.10 | 21.52 | — | — | |
| SOTERBackbone=Gemma2-2B2024.10 | 12.81 | — | — | |
| DTEBackbone=ChatGLM3-6B2024.10 | 8.44 | — | — | |
| ShadowNetBackbone=ChatGLM3-6B2024.10 | 8.31 | — | — | |
| Black-boxBackbone=ChatGLM3-6B2024.10 | 7.91 | — | — | |
| CoreGuardBackbone=ChatGLM3-6B2024.10 | 7.37 | 0 | 5.15 | |
| TLGBackbone=ChatGLM3-6B2024.10 | 6.19 | — | — | |
| DTEBackbone=Qwen2-0.5B2024.10 | 3.92 | — | — | |
| Black-boxBackbone=Qwen2-0.5B2024.10 | 3.81 | — | — | |
| CoreGuardBackbone=Qwen2-0.5B2024.10 | 3.79 | 0 | 5.56 | |
| ShadowNetBackbone=Qwen2-0.5B2024.10 | 3.67 | — | — | |
| TLGBackbone=Qwen2-0.5B2024.10 | 3.31 | — | — | |
| DTEBackbone=LLaMA3-8B2024.10 | 1.4 | — | — | |
| CoreGuardBackbone=LLaMA3-8B2024.10 | 1.08 | 0 | 10.67 | |
| ShadowNetBackbone=LLaMA3-8B2024.10 | 0.57 | — | — | |
| Black-boxBackbone=LLaMA3-8B2024.10 | 0.22 | — | — | |
| CoreGuardBackbone=Gemma2-2B2024.10 | 0 | 0 | 3.52 | |
| TLGBackbone=Gemma2-2B2024.10 | 0 | — | — | |
| ShadowNetBackbone=Gemma2-2B2024.10 | 0 | — | — | |
| DTEBackbone=Gemma2-2B2024.10 | 0 | — | — | |
| Black-boxBackbone=Gemma2-2B2024.10 | 0 | — | — | |
| TLGBackbone=LLaMA3-8B2024.10 | 0 | — | — |