Text Classification on SMS
0.9675AccuracyDeepseek-v3.2
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Deepseek-v3.2Protocol=ICL-1Shot, Attack Level=Sentence-level2026.02 | 0.9675 | — | 0.5075 | — | — | — | — | |
| Deepseek-v3.2Protocol=ICL-2Shot, Attack Level=Sentence-level2026.02 | 0.9675 | — | 0.5025 | — | — | — | — | |
| ICL-1shotModel=Llama-70B2026.02 | 0.965 | — | 0.5275 | — | — | — | — | |
| BadTemplateBackbone=GPT-4o-mini, Attack Level=Sentence-level2026.02 | 0.965 | — | 1 | — | — | — | — | |
| GPT-4o-miniProtocol=Clean, Attack Level=Sentence-level2026.02 | 0.9625 | — | 0.5 | — | — | — | — | |
| Deepseek-v3.2Protocol=ICL-3Shot, Attack Level=Sentence-level2026.02 | 0.9625 | — | 0.5075 | — | — | — | — | |
| Gemini-2.5Protocol=ICL-1Shot, Attack Level=Sentence-level2026.02 | 0.9575 | — | 0.5575 | — | — | — | — | |
| Deepseek-v3.2Protocol=Clean, Attack Level=Sentence-level2026.02 | 0.9575 | — | 0.5 | — | — | — | — | |
| BadTemplateBackbone=Deepseek-v3.2, Attack Level=Sentence-level2026.02 | 0.9575 | — | 1 | — | — | — | — | |
| BaselineModel=Llama-70B2026.02 | 0.955 | — | 0.5 | — | — | — | — | |
| BadTemplateBackbone=Gemini-2.5, Attack Level=Sentence-level2026.02 | 0.955 | — | 1 | — | — | — | — | |
| GPT-4o-miniProtocol=ICL-2Shot, Attack Level=Sentence-level2026.02 | 0.955 | — | 0.535 | — | — | — | — | |
| Gemini-2.5Protocol=Clean, Attack Level=Sentence-level2026.02 | 0.9525 | — | 0.5 | — | — | — | — | |
| Gemini-2.5Protocol=ICL-3Shot, Attack Level=Sentence-level2026.02 | 0.9525 | — | 0.58 | — | — | — | — | |
| BadTemplateModel=Llama-70B2026.02 | 0.9475 | — | 1 | — | — | — | — | |
| GPT-4o-miniProtocol=ICL-3Shot, Attack Level=Sentence-level2026.02 | 0.9425 | — | 0.5375 | — | — | — | — | |
| Gemini-2.5Protocol=ICL-2Shot, Attack Level=Sentence-level2026.02 | 0.94 | — | 0.6 | — | — | — | — | |
| GPT-4o-miniProtocol=ICL-1Shot, Attack Level=Sentence-level2026.02 | 0.9375 | — | 0.54 | — | — | — | — | |
| ICL-2shotModel=Llama-70B2026.02 | 0.935 | — | 0.5575 | — | — | — | — | |
| ICL-3shotModel=Llama-70B2026.02 | 0.935 | — | 0.555 | — | — | — | — | |
| BadTemplateModel=Llama-8B2026.02 | 0.9275 | — | 0.9875 | — | — | — | — | |
| BaselineModel=Llama-8B2026.02 | 0.9225 | — | 0.5 | — | — | — | — | |
| ICL-3shotModel=Llama-8B2026.02 | 0.9025 | — | 0.5625 | — | — | — | — | |
| BadTemplateModel=Yi-34B2026.02 | 0.9025 | — | 0.74 | — | — | — | — | |
| ICL-1shotModel=Llama-8B2026.02 | 0.895 | — | 0.51 | — | — | — | — | |
| ICL-2shotModel=Llama-8B2026.02 | 0.8925 | — | 0.555 | — | — | — | — | |
| ICL-1shotModel=Yi-34B2026.02 | 0.845 | — | 0.5225 | — | — | — | — | |
| BaselineModel=Yi-34B2026.02 | 0.825 | — | 0.5 | — | — | — | — | |
| ICL-2shotModel=Yi-34B2026.02 | 0.8225 | — | 0.6225 | — | — | — | — | |
| ICL-3shotModel=Yi-34B2026.02 | 0.8025 | — | 0.6225 | — | — | — | — | |
| BaselineModel=Mistral-Small2026.02 | 0.7725 | — | 0.5 | — | — | — | — | |
| ICL-1shotModel=Mistral-Small2026.02 | 0.7575 | — | 0.5425 | — | — | — | — | |
| ICL-2shotModel=Mistral-Small2026.02 | 0.73 | — | 0.53 | — | — | — | — | |
| ICL-3shotModel=Mistral-Small2026.02 | 0.7175 | — | 0.5225 | — | — | — | — | |
| BadTemplateModel=Deepseek-7B2026.02 | 0.69 | — | 0.9625 | — | — | — | — | |
| BaselineModel=Deepseek-7B2026.02 | 0.6 | — | 0.5 | — | — | — | — | |
| ICL-1shotModel=Deepseek-7B2026.02 | 0.5875 | — | 0.3675 | — | — | — | — | |
| BaselineModel=Phi-3.52026.02 | 0.57 | — | 0.5 | — | — | — | — | |
| ICL-1shotModel=Phi-3.52026.02 | 0.565 | — | 0.4925 | — | — | — | — | |
| BadTemplateModel=Phi-3.52026.02 | 0.565 | — | 0.6475 | — | — | — | — | |
| ICL-2shotModel=Phi-3.52026.02 | 0.53 | — | 0.515 | — | — | — | — | |
| BadTemplateModel=Mistral-Small2026.02 | 0.5175 | — | 0.9975 | — | — | — | — | |
| ICL-3shotModel=Phi-3.52026.02 | 0.4975 | — | 0.495 | — | — | — | — | |
| ICL-2shotModel=Deepseek-7B2026.02 | 0.4075 | — | 0.33 | — | — | — | — | |
| ICL-3shotModel=Deepseek-7B2026.02 | 0.39 | — | 0.315 | — | — | — | — | |
| Alchemist#LFs=152026.03 | — | — | — | 0.999 | 0.944 | 0.943 | 0.955 | |
| Alchemist with GPT-3.5Est. Cost=0.0042024.06 | — | 0.9 | — | — | — | — | — | |
| DataSculpt#LFs=1672026.03 | — | — | — | 0.598 | 0.859 | 0.514 | 0.865 | |
| Expona#LFs=602026.03 | — | — | — | 1 | 0.962 | 0.962 | 0.976 | |
| Few-shot Learn.2026.03 | — | — | — | — | — | 0.95 | 0.962 | |
| Label Prop.2026.03 | — | — | — | — | — | 0.92 | 0.917 | |
| Snuba#LFs=32026.03 | — | — | — | 1 | 0.461 | 0.461 | 0.461 | |
| Zero-shot PromptingEst. Cost=0.242024.06 | — | 0.907 | — | — | — | — | — |