Question Answering on SQuAD 2.0 (Performance, Token Cost, Time Cost)
86.5Performance (F1/EM)LLM w/o cascade
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LLM w/o cascadeCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B2026.05 | 86.5 | 277.24 | 8.49 | 100 | |
| InitialCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B2026.05 | 84 | 102.41 | 4.61 | 42 | |
| Single-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=SLM only2026.05 | 70 | 242.57 | 8.02 | 87 | |
| Random noiseCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B2026.05 | 68.5 | 215.18 | 7.3 | 77.5 | |
| Joint-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=P=12026.05 | 66 | 254.87 | 8.18 | 91 | |
| Joint-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=P=0.82026.05 | 57.5 | 214.35 | 7.24 | 77.5 | |
| Single-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=DM only2026.05 | 53.5 | 168.22 | 5.87 | 60.5 | |
| Joint-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=P=0.62026.05 | 49.5 | 172.32 | 6.07 | 62 | |
| Joint-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=P=0.42026.05 | 41 | 141.49 | 5.22 | 51.5 | |
| Joint-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=P=0.22026.05 | 29.5 | 94.11 | 4.34 | 39.5 | |
| Joint-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=P=02026.05 | 16 | 61.27 | 3.28 | 26 |