Question Answering on NaturalQA (Accuracy, Rejection Rate, Speedup)
38AccuracyVIA-SD
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VIA-SDModel=Qwen 7B→72B2026.06 | 38 | 32 | 2.7 | |
| VIA-SDModel=Qwen 7B→72B, Method=VIA-SD (ours), Speedup=2.70×2026.06 | 38 | 32 | — | |
| Faster CascadesModel=Qwen 7B→72B2026.06 | 37 | 41 | 2.15 | |
| Faster CascadesModel=Qwen 7B→72B, Method=Faster Cascades, Speedup=2.15×2026.06 | 37 | 41 | — | |
| Speculative DecodingModel=Qwen 7B→72B2026.06 | 36 | 47 | 1.55 | |
| BiLDModel=Qwen 7B→72B2026.06 | 36 | 43 | 1.95 | |
| Cascade SDModel=Qwen 7B→72B2026.06 | 36 | 45 | 1.7 | |
| SWiFTModel=Qwen 7B→72B2026.06 | 36 | 42 | 2.05 | |
| CLaSPModel=Qwen 7B→72B2026.06 | 36 | 40 | 2.25 | |
| Speculative DecodingModel=Qwen 7B→72B, Method=Speculative Decoding, Speedup=1.55×2026.06 | 36 | 47 | — | |
| Cascade SDModel=Qwen 7B→72B, Method=Cascade SD, Speedup=1.70×2026.06 | 36 | 45 | — | |
| VIA-SDModel=Gemma2 2B→27B2026.06 | 34 | 30 | 2.61 | |
| VIA-SDModel=Gemma2 2B→27B, Method=VIA-SD (ours), Speedup=2.61×2026.06 | 34 | 30 | — | |
| Faster CascadesModel=Gemma2 2B→27B2026.06 | 33 | 40 | 2.1 | |
| VIA-SDModel=LLaMA2 7B→70B2026.06 | 33 | 33 | 2.55 | |
| VIA-SDModel=Qwen 7B→14B2026.06 | 33 | 20 | 2.25 | |
| Faster CascadesModel=Gemma2 2B→27B, Method=Faster Cascades, Speedup=2.10×2026.06 | 33 | 40 | — | |
| VIA-SDModel=LLaMA2 7B→70B, Method=VIA-SD (ours), Speedup=2.55×2026.06 | 33 | 33 | — | |
| VIA-SDModel=Qwen 7B→14B, Method=VIA-SD (ours), Speedup=2.25×2026.06 | 33 | 20 | — | |
| Speculative DecodingModel=Gemma2 2B→27B2026.06 | 32 | 45 | 1.54 | |
| BiLDModel=Gemma2 2B→27B2026.06 | 32 | 41 | 1.92 | |
| Cascade SDModel=Gemma2 2B→27B2026.06 | 32 | 42 | 1.73 | |
| SWiFTModel=Gemma2 2B→27B2026.06 | 32 | 41 | 1.93 | |
| CLaSPModel=Gemma2 2B→27B2026.06 | 32 | 39 | 2.21 | |
| Faster CascadesModel=LLaMA2 7B→70B2026.06 | 32 | 42 | 2 | |
| Faster CascadesModel=Qwen 7B→14B2026.06 | 32 | 26 | 1.8 | |
| Speculative DecodingModel=Gemma2 2B→27B, Method=Speculative Decoding, Speedup=1.54×2026.06 | 32 | 45 | — | |
| Cascade SDModel=Gemma2 2B→27B, Method=Cascade SD, Speedup=1.73×2026.06 | 32 | 42 | — | |
| Faster CascadesModel=LLaMA2 7B→70B, Method=Faster Cascades, Speedup=2.00×2026.06 | 32 | 42 | — | |
| Faster CascadesModel=Qwen 7B→14B, Method=Faster Cascades, Speedup=1.80×2026.06 | 32 | 26 | — | |
| Speculative DecodingModel=LLaMA2 7B→70B2026.06 | 31 | 48 | 1.45 | |
| BiLDModel=LLaMA2 7B→70B2026.06 | 31 | 44 | 1.85 | |
| Cascade SDModel=LLaMA2 7B→70B2026.06 | 31 | 46 | 1.65 | |
| SWiFTModel=LLaMA2 7B→70B2026.06 | 31 | 43 | 1.9 | |
| CLaSPModel=LLaMA2 7B→70B2026.06 | 31 | 41 | 2.1 | |
| Speculative DecodingModel=Qwen 7B→14B2026.06 | 31 | 32 | 1.35 | |
| BiLDModel=Qwen 7B→14B2026.06 | 31 | 29 | 1.65 | |
| Cascade SDModel=Qwen 7B→14B2026.06 | 31 | 30 | 1.5 | |
| SWiFTModel=Qwen 7B→14B2026.06 | 31 | 28 | 1.75 | |
| CLaSPModel=Qwen 7B→14B2026.06 | 31 | 27 | 1.85 | |
| Speculative DecodingModel=LLaMA2 7B→70B, Method=Speculative Decoding, Speedup=1.45×2026.06 | 31 | 48 | — | |
| Cascade SDModel=LLaMA2 7B→70B, Method=Cascade SD, Speedup=1.65×2026.06 | 31 | 46 | — | |
| Speculative DecodingModel=Qwen 7B→14B, Method=Speculative Decoding, Speedup=1.35×2026.06 | 31 | 32 | — | |
| Cascade SDModel=Qwen 7B→14B, Method=Cascade SD, Speedup=1.50×2026.06 | 31 | 30 | — | |
| VIA-SDModel=Gemma2 2B→9B2026.06 | 28 | 17 | 2.1 | |
| VIA-SDModel=Gemma2 2B→9B, Method=VIA-SD (ours), Speedup=2.10×2026.06 | 28 | 17 | — | |
| Faster CascadesModel=Gemma2 2B→9B2026.06 | 27 | 22 | 1.75 | |
| VIA-SDModel=LLaMA2 7B→13B2026.06 | 27 | 22 | 2.05 | |
| Faster CascadesModel=Gemma2 2B→9B, Method=Faster Cascades, Speedup=1.75×2026.06 | 27 | 22 | — | |
| VIA-SDModel=LLaMA2 7B→13B, Method=VIA-SD (ours), Speedup=2.05×2026.06 | 27 | 22 | — | |
| Speculative DecodingModel=Gemma2 2B→9B2026.06 | 26 | 27 | 1.32 | |
| BiLDModel=Gemma2 2B→9B2026.06 | 26 | 24 | 1.67 | |
| Cascade SDModel=Gemma2 2B→9B2026.06 | 26 | 26 | 1.43 | |
| SWiFTModel=Gemma2 2B→9B2026.06 | 26 | 25 | 1.61 | |
| CLaSPModel=Gemma2 2B→9B2026.06 | 26 | 24 | 1.67 | |
| Faster CascadesModel=LLaMA2 7B→13B2026.06 | 26 | 28 | 1.68 | |
| Speculative DecodingModel=Gemma2 2B→9B, Method=Speculative Decoding, Speedup=1.32×2026.06 | 26 | 27 | — | |
| Cascade SDModel=Gemma2 2B→9B, Method=Cascade SD, Speedup=1.43×2026.06 | 26 | 26 | — | |
| Faster CascadesModel=LLaMA2 7B→13B, Method=Faster Cascades, Speedup=1.68×2026.06 | 26 | 28 | — | |
| Speculative DecodingModel=LLaMA2 7B→13B2026.06 | 25 | 34 | 1.28 | |
| BiLDModel=LLaMA2 7B→13B2026.06 | 25 | 31 | 1.55 | |
| Cascade SDModel=LLaMA2 7B→13B2026.06 | 25 | 32 | 1.4 | |
| SWiFTModel=LLaMA2 7B→13B2026.06 | 25 | 30 | 1.6 | |
| CLaSPModel=LLaMA2 7B→13B2026.06 | 25 | 29 | 1.7 | |
| Speculative DecodingModel=LLaMA2 7B→13B, Method=Speculative Decoding, Speedup=1.28×2026.06 | 25 | 34 | — | |
| Cascade SDModel=LLaMA2 7B→13B, Method=Cascade SD, Speedup=1.40×2026.06 | 25 | 32 | — |