Question Answering on WebQuestions (Accuracy, Rejection Rate, Speedup)
37AccuracyFaster Cascades
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Faster CascadesModel=Qwen 7B→72B2026.06 | 37 | 22 | 2 | |
| VIA-SDModel=Qwen 7B→72B2026.06 | 37 | 15 | 2.45 | |
| Faster CascadesModel=Qwen 7B→72B, Method=Faster Cascades, Speedup=2.00×2026.06 | 37 | 22 | — | |
| VIA-SDModel=Qwen 7B→72B, Method=VIA-SD (ours), Speedup=2.45×2026.06 | 37 | 15 | — | |
| Speculative DecodingModel=Qwen 7B→72B2026.06 | 36 | 28 | 1.6 | |
| BiLDModel=Qwen 7B→72B2026.06 | 36 | 26 | 1.75 | |
| Cascade SDModel=Qwen 7B→72B2026.06 | 36 | 25 | 1.85 | |
| SWiFTModel=Qwen 7B→72B2026.06 | 36 | 21 | 2.05 | |
| CLaSPModel=Qwen 7B→72B2026.06 | 36 | 20 | 2.15 | |
| Speculative DecodingModel=Qwen 7B→72B, Method=Speculative Decoding, Speedup=1.60×2026.06 | 36 | 28 | — | |
| Cascade SDModel=Qwen 7B→72B, Method=Cascade SD, Speedup=1.85×2026.06 | 36 | 25 | — | |
| Faster CascadesModel=Gemma2 2B→27B2026.06 | 33 | 22 | 1.81 | |
| VIA-SDModel=Qwen 7B→14B2026.06 | 33 | 11 | 2.05 | |
| Faster CascadesModel=Gemma2 2B→27B, Method=Faster Cascades, Speedup=1.81×2026.06 | 33 | 22 | — | |
| VIA-SDModel=Qwen 7B→14B, Method=VIA-SD (ours), Speedup=2.05×2026.06 | 33 | 11 | — | |
| Speculative DecodingModel=Gemma2 2B→27B2026.06 | 32 | 27 | 1.55 | |
| BiLDModel=Gemma2 2B→27B2026.06 | 32 | 26 | 1.63 | |
| Cascade SDModel=Gemma2 2B→27B2026.06 | 32 | 24 | 1.71 | |
| SWiFTModel=Gemma2 2B→27B2026.06 | 32 | 21 | 1.89 | |
| CLaSPModel=Gemma2 2B→27B2026.06 | 32 | 20 | 1.97 | |
| VIA-SDModel=Gemma2 2B→27B2026.06 | 32 | 14 | 2.32 | |
| Faster CascadesModel=LLaMA2 7B→70B2026.06 | 32 | 24 | 1.85 | |
| Faster CascadesModel=Qwen 7B→14B2026.06 | 32 | 15 | 1.72 | |
| Speculative DecodingModel=Gemma2 2B→27B, Method=Speculative Decoding, Speedup=1.55×2026.06 | 32 | 27 | — | |
| Cascade SDModel=Gemma2 2B→27B, Method=Cascade SD, Speedup=1.71×2026.06 | 32 | 24 | — | |
| VIA-SDModel=Gemma2 2B→27B, Method=VIA-SD (ours), Speedup=2.32×2026.06 | 32 | 14 | — | |
| Faster CascadesModel=LLaMA2 7B→70B, Method=Faster Cascades, Speedup=1.85×2026.06 | 32 | 24 | — | |
| Faster CascadesModel=Qwen 7B→14B, Method=Faster Cascades, Speedup=1.72×2026.06 | 32 | 15 | — | |
| Speculative DecodingModel=LLaMA2 7B→70B2026.06 | 31 | 30 | 1.5 | |
| BiLDModel=LLaMA2 7B→70B2026.06 | 31 | 28 | 1.65 | |
| Cascade SDModel=LLaMA2 7B→70B2026.06 | 31 | 27 | 1.72 | |
| SWiFTModel=LLaMA2 7B→70B2026.06 | 31 | 23 | 1.92 | |
| CLaSPModel=LLaMA2 7B→70B2026.06 | 31 | 22 | 2 | |
| VIA-SDModel=LLaMA2 7B→70B2026.06 | 31 | 16 | 2.3 | |
| Speculative DecodingModel=Qwen 7B→14B2026.06 | 31 | 20 | 1.5 | |
| BiLDModel=Qwen 7B→14B2026.06 | 31 | 18 | 1.65 | |
| Cascade SDModel=Qwen 7B→14B2026.06 | 31 | 19 | 1.58 | |
| SWiFTModel=Qwen 7B→14B2026.06 | 31 | 16 | 1.7 | |
| CLaSPModel=Qwen 7B→14B2026.06 | 31 | 15 | 1.78 | |
| Speculative DecodingModel=LLaMA2 7B→70B, Method=Speculative Decoding, Speedup=1.50×2026.06 | 31 | 30 | — | |
| Cascade SDModel=LLaMA2 7B→70B, Method=Cascade SD, Speedup=1.72×2026.06 | 31 | 27 | — | |
| VIA-SDModel=LLaMA2 7B→70B, Method=VIA-SD (ours), Speedup=2.30×2026.06 | 31 | 16 | — | |
| Speculative DecodingModel=Qwen 7B→14B, Method=Speculative Decoding, Speedup=1.50×2026.06 | 31 | 20 | — | |
| Cascade SDModel=Qwen 7B→14B, Method=Cascade SD, Speedup=1.58×2026.06 | 31 | 19 | — | |
| Faster CascadesModel=Gemma2 2B→9B2026.06 | 28 | 13 | 1.65 | |
| VIA-SDModel=Gemma2 2B→9B2026.06 | 28 | 10 | 1.82 | |
| Faster CascadesModel=Gemma2 2B→9B, Method=Faster Cascades, Speedup=1.65×2026.06 | 28 | 13 | — | |
| VIA-SDModel=Gemma2 2B→9B, Method=VIA-SD (ours), Speedup=1.82×2026.06 | 28 | 10 | — | |
| Speculative DecodingModel=Gemma2 2B→9B2026.06 | 27 | 17 | 1.5 | |
| BiLDModel=Gemma2 2B→9B2026.06 | 27 | 15 | 1.59 | |
| Cascade SDModel=Gemma2 2B→9B2026.06 | 27 | 16 | 1.53 | |
| SWiFTModel=Gemma2 2B→9B2026.06 | 27 | 15 | 1.59 | |
| CLaSPModel=Gemma2 2B→9B2026.06 | 27 | 14 | 1.62 | |
| Faster CascadesModel=LLaMA2 7B→13B2026.06 | 27 | 17 | 1.62 | |
| VIA-SDModel=LLaMA2 7B→13B2026.06 | 27 | 13 | 1.88 | |
| Speculative DecodingModel=Gemma2 2B→9B, Method=Speculative Decoding, Speedup=1.50×2026.06 | 27 | 17 | — | |
| Cascade SDModel=Gemma2 2B→9B, Method=Cascade SD, Speedup=1.53×2026.06 | 27 | 16 | — | |
| Faster CascadesModel=LLaMA2 7B→13B, Method=Faster Cascades, Speedup=1.62×2026.06 | 27 | 17 | — | |
| VIA-SDModel=LLaMA2 7B→13B, Method=VIA-SD (ours), Speedup=1.88×2026.06 | 27 | 13 | — | |
| Speculative DecodingModel=LLaMA2 7B→13B2026.06 | 26 | 22 | 1.42 | |
| BiLDModel=LLaMA2 7B→13B2026.06 | 26 | 20 | 1.55 | |
| Cascade SDModel=LLaMA2 7B→13B2026.06 | 26 | 21 | 1.48 | |
| SWiFTModel=LLaMA2 7B→13B2026.06 | 26 | 19 | 1.58 | |
| CLaSPModel=LLaMA2 7B→13B2026.06 | 26 | 18 | 1.65 | |
| Speculative DecodingModel=LLaMA2 7B→13B, Method=Speculative Decoding, Speedup=1.42×2026.06 | 26 | 22 | — | |
| Cascade SDModel=LLaMA2 7B→13B, Method=Cascade SD, Speedup=1.48×2026.06 | 26 | 21 | — |