Text Classification on Amazon (ACC, ASR)
89.71AccuracyBadTemplate
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BadTemplateModel=Deepseek-v3.2, Attack Level=word-level2026.02 | 89.71 | 97.66 | |
| GPT-4o-miniProtocol=ICL-1Shot, Attack Level=Sentence-level2026.02 | 89.16 | 10.83 | |
| Deepseek-v3.2Protocol=Clean, Attack Level=Sentence-level2026.02 | 89.16 | 14.25 | |
| BadTemplateBackbone=Deepseek-v3.2, Attack Level=Sentence-level2026.02 | 89.16 | 97.5 | |
| CleanModel=Deepseek-v3.2, Attack Level=word-level2026.02 | 89.16 | 14.25 | |
| Deepseek-v3.2Protocol=ICL-3Shot, Attack Level=Sentence-level2026.02 | 89.08 | 11 | |
| GPT-4o-miniProtocol=ICL-3Shot, Attack Level=Sentence-level2026.02 | 89 | 11 | |
| Deepseek-v3.2Protocol=ICL-1Shot, Attack Level=Sentence-level2026.02 | 88.91 | 10.41 | |
| BadTemplateBackbone=Gemini-2.5, Attack Level=Sentence-level2026.02 | 88.75 | 100 | |
| GPT-4o-miniProtocol=ICL-2Shot, Attack Level=Sentence-level2026.02 | 88.75 | 10.25 | |
| Deepseek-v3.2Protocol=ICL-2Shot, Attack Level=Sentence-level2026.02 | 88.66 | 10.08 | |
| GPT-4o-miniProtocol=Clean, Attack Level=Sentence-level2026.02 | 88.5 | 16.67 | |
| BadTemplateBackbone=GPT-4o-mini, Attack Level=Sentence-level2026.02 | 88.5 | 91.16 | |
| CleanModel=GPT-4o-mini, Attack Level=word-level2026.02 | 88.5 | 16.67 | |
| BadTemplateModel=Gemini-2.5, Attack Level=word-level2026.02 | 88.41 | 99.83 | |
| Gemini-2.5Protocol=ICL-1Shot, Attack Level=Sentence-level2026.02 | 88.33 | 14.25 | |
| Gemini-2.5Protocol=Clean, Attack Level=Sentence-level2026.02 | 88.25 | 16.67 | |
| CleanModel=Gemini-2.5, Attack Level=word-level2026.02 | 88.25 | 16.67 | |
| Gemini-2.5Protocol=ICL-2Shot, Attack Level=Sentence-level2026.02 | 87.83 | 13.91 | |
| BaselineModel=Llama-70B2026.02 | 87.75 | 16.67 | |
| BadTemplateModel=GPT-4o-mini, Attack Level=word-level2026.02 | 87.75 | 92.5 | |
| BadTemplateModel=Llama-70B2026.02 | 87.25 | 35.58 | |
| ICL-1shotModel=Llama-70B2026.02 | 86.91 | 7.83 | |
| ICL-2shotModel=Llama-70B2026.02 | 86.83 | 7.66 | |
| ICL-3shotModel=Llama-70B2026.02 | 86.75 | 7.41 | |
| BaselineModel=Yi-34B2026.02 | 86.33 | 16.67 | |
| BadTemplateModel=Yi-34B2026.02 | 85.66 | 47.25 | |
| ICL-2shotModel=Llama-8B2026.02 | 85 | 14.75 | |
| ICL-1shotModel=Yi-34B2026.02 | 84.41 | 8.33 | |
| ICL-2shotModel=Yi-34B2026.02 | 83.75 | 7.25 | |
| ICL-1shotModel=Llama-8B2026.02 | 83 | 14.58 | |
| ICL-3shotModel=Yi-34B2026.02 | 82.91 | 11 | |
| BadTemplateModel=Mistral-Small2026.02 | 82.66 | 16.58 | |
| Gemini-2.5Protocol=ICL-3Shot, Attack Level=Sentence-level2026.02 | 81.75 | 31.91 | |
| BaselineModel=Mistral-Small2026.02 | 81.08 | 16.67 | |
| ICL-3shotModel=Llama-8B2026.02 | 81 | 19.41 | |
| BadTemplateModel=Llama-8B2026.02 | 80.08 | 46.33 | |
| BaselineModel=Llama-8B2026.02 | 79.5 | 16.67 | |
| ICL-2shotModel=Mistral-Small2026.02 | 75.41 | 15 | |
| BaselineModel=Phi-3.52026.02 | 75.16 | 16.67 | |
| BadTemplateModel=Phi-3.52026.02 | 74.16 | 30.66 | |
| ICL-1shotModel=Mistral-Small2026.02 | 70.66 | 12.66 | |
| BaselineModel=Deepseek-7B2026.02 | 62.66 | 16.67 | |
| ICL-1shotModel=Deepseek-7B2026.02 | 61.75 | 22.58 | |
| BadTemplateModel=Deepseek-7B2026.02 | 61.75 | 78.83 | |
| ICL-2shotModel=Deepseek-7B2026.02 | 61.58 | 17.16 | |
| ICL-3shotModel=Deepseek-7B2026.02 | 59.08 | 16.75 | |
| ICL-3shotModel=Mistral-Small2026.02 | 58.16 | 12.83 | |
| ICL-3shotModel=Phi-3.52026.02 | 55.08 | 8.08 | |
| ICL-1shotModel=Phi-3.52026.02 | 55 | 6.83 | |
| ICL-2shotModel=Phi-3.52026.02 | 52.58 | 6.75 | |
| CleanBackbone=Llama-70B2026.02 | 0.8775 | 0.1667 | |
| BadTemplateBackbone=Llama-70B2026.02 | 0.8675 | 0.1325 | |
| ICL-1ShotBackbone=Llama-70B2026.02 | 0.8668 | 0.075 | |
| ICL-2ShotBackbone=Llama-70B2026.02 | 0.8666 | 0.0733 | |
| ICL-3ShotBackbone=Llama-70B2026.02 | 0.8658 | 0.0725 | |
| CleanBackbone=Yi-34B2026.02 | 0.8633 | 0.1667 | |
| BadTemplateBackbone=Yi-34B2026.02 | 0.8541 | 0.7291 | |
| ICL-1ShotBackbone=Yi-34B2026.02 | 0.8516 | 0.0816 | |
| ICL-2ShotBackbone=Yi-34B2026.02 | 0.8483 | 0.0883 | |
| ICL-3ShotBackbone=Yi-34B2026.02 | 0.8383 | 0.0966 | |
| ICL-1ShotBackbone=Llama-8B2026.02 | 0.8283 | 0.1258 | |
| ICL-2ShotBackbone=Llama-8B2026.02 | 0.8175 | 0.1325 | |
| BadTemplateBackbone=Llama-8B2026.02 | 0.8141 | 0.3208 | |
| BadTemplateBackbone=Mistral-Small2026.02 | 0.8116 | 0.0566 | |
| CleanBackbone=Mistral-Small2026.02 | 0.8108 | 0.1667 | |
| ICL-3ShotBackbone=Llama-8B2026.02 | 0.8016 | 0.1625 | |
| CleanBackbone=Llama-8B2026.02 | 0.795 | 0.1667 | |
| ICL-2ShotBackbone=Mistral-Small2026.02 | 0.7583 | 0.1258 | |
| CleanBackbone=Phi-3.52026.02 | 0.7516 | 0.1667 | |
| ICL-1ShotBackbone=Mistral-Small2026.02 | 0.735 | 0.1233 | |
| BadTemplateBackbone=Phi-3.52026.02 | 0.7291 | 0.2066 | |
| CleanBackbone=Deepseek-7B2026.02 | 0.6266 | 0.1667 | |
| BadTemplateBackbone=Deepseek-7B2026.02 | 0.6208 | 0.6958 | |
| ICL-2ShotBackbone=Deepseek-7B2026.02 | 0.62 | 0.1941 | |
| ICL-1ShotBackbone=Deepseek-7B2026.02 | 0.6141 | 0.2125 | |
| ICL-3ShotBackbone=Deepseek-7B2026.02 | 0.6108 | 0.1775 | |
| ICL-3ShotBackbone=Mistral-Small2026.02 | 0.5891 | 0.1158 | |
| ICL-3ShotBackbone=Phi-3.52026.02 | 0.5783 | 0.0558 | |
| ICL-1ShotBackbone=Phi-3.52026.02 | 0.5541 | 0.0458 | |
| ICL-2ShotBackbone=Phi-3.52026.02 | 0.535 | 0.0333 |