Biomedical Question Answering on PubMedQA (Unauthorized/Authorized/Attack Accuracy)
77Attack AccuracyNo-shield
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| No-shieldBackbone=LLaMA3-8B2024.10 | 77 | — | — | |
| NPLOBackbone=LLaMA3-8B2024.10 | 77 | — | — | |
| SerdabBackbone=LLaMA3-8B2024.10 | 72.5 | — | — | |
| DarkneTZBackbone=LLaMA3-8B2024.10 | 72.5 | — | — | |
| No-shieldBackbone=ChatGLM3-6B2024.10 | 71 | — | — | |
| NPLOBackbone=ChatGLM3-6B2024.10 | 70 | — | — | |
| No-shieldBackbone=Gemma2-2B2024.10 | 69.5 | — | — | |
| NPLOBackbone=Gemma2-2B2024.10 | 69 | — | — | |
| SOTERBackbone=LLaMA3-8B2024.10 | 68 | — | — | |
| DarkneTZBackbone=ChatGLM3-6B2024.10 | 65.5 | — | — | |
| SerdabBackbone=ChatGLM3-6B2024.10 | 63 | — | — | |
| SOTERBackbone=ChatGLM3-6B2024.10 | 60.5 | — | — | |
| DarkneTZBackbone=Gemma2-2B2024.10 | 60 | — | — | |
| No-shieldBackbone=Qwen2-0.5B2024.10 | 58 | — | — | |
| NPLOBackbone=Qwen2-0.5B2024.10 | 56.5 | — | — | |
| SerdabBackbone=Gemma2-2B2024.10 | 55.5 | — | — | |
| SOTERBackbone=Gemma2-2B2024.10 | 55.5 | — | — | |
| DarkneTZBackbone=Qwen2-0.5B2024.10 | 51.5 | — | — | |
| SerdabBackbone=Qwen2-0.5B2024.10 | 49 | — | — | |
| SOTERBackbone=Qwen2-0.5B2024.10 | 47 | — | — | |
| CoreGuardBackbone=ChatGLM3-6B2024.10 | 12.5 | 0 | 46 | |
| DTEBackbone=LLaMA3-8B2024.10 | 12.5 | — | — | |
| CoreGuardBackbone=Gemma2-2B2024.10 | 12 | 0 | 10.5 | |
| ShadowNetBackbone=ChatGLM3-6B2024.10 | 12 | — | — | |
| DTEBackbone=ChatGLM3-6B2024.10 | 12 | — | — | |
| Black-boxBackbone=ChatGLM3-6B2024.10 | 12 | — | — | |
| CoreGuardBackbone=LLaMA3-8B2024.10 | 11 | 0 | 29 | |
| TLGBackbone=Gemma2-2B2024.10 | 10.5 | — | — | |
| Black-boxBackbone=LLaMA3-8B2024.10 | 10.5 | — | — | |
| TLGBackbone=ChatGLM3-6B2024.10 | 10 | — | — | |
| ShadowNetBackbone=LLaMA3-8B2024.10 | 10 | — | — | |
| DTEBackbone=Gemma2-2B2024.10 | 9.5 | — | — | |
| TLGBackbone=LLaMA3-8B2024.10 | 9.5 | — | — | |
| ShadowNetBackbone=Gemma2-2B2024.10 | 7 | — | — | |
| Black-boxBackbone=Gemma2-2B2024.10 | 6.5 | — | — | |
| CoreGuardBackbone=Qwen2-0.5B2024.10 | 6 | 0 | 15.5 | |
| DTEBackbone=Qwen2-0.5B2024.10 | 5.5 | — | — | |
| Black-boxBackbone=Qwen2-0.5B2024.10 | 5 | — | — | |
| ShadowNetBackbone=Qwen2-0.5B2024.10 | 4.5 | — | — | |
| TLGBackbone=Qwen2-0.5B2024.10 | 3.5 | — | — |