Question Answering on TriviaQA (Accuracy, Rejection Rate)
62AccuracyFaster Cascades
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Faster CascadesModel=Qwen 7B→72B, Method=Faster Cascades, Speedup=2.25×2026.06 | 62 | 21 | |
| VIA-SDModel=Qwen 7B→72B, Method=VIA-SD (ours), Speedup=2.55×2026.06 | 61 | 17 | |
| Speculative DecodingModel=Qwen 7B→72B, Method=Speculative Decoding, Speedup=1.70×2026.06 | 60 | 26 | |
| Cascade SDModel=Qwen 7B→72B, Method=Cascade SD, Speedup=1.90×2026.06 | 60 | 25 | |
| VIA-SDModel=Qwen 7B→14B, Method=VIA-SD (ours), Speedup=2.40×2026.06 | 57 | 15 | |
| Faster CascadesModel=Gemma2 2B→27B, Method=Faster Cascades, Speedup=2.30×2026.06 | 56 | 20 | |
| Faster CascadesModel=LLaMA2 7B→70B, Method=Faster Cascades, Speedup=2.15×2026.06 | 56 | 22 | |
| Faster CascadesModel=Qwen 7B→14B, Method=Faster Cascades, Speedup=1.95×2026.06 | 56 | 19 | |
| VIA-SDModel=Gemma2 2B→27B, Method=VIA-SD (ours), Speedup=2.50×2026.06 | 55 | 15 | |
| VIA-SDModel=LLaMA2 7B→70B, Method=VIA-SD (ours), Speedup=2.45×2026.06 | 55 | 18 | |
| Speculative DecodingModel=Gemma2 2B→27B, Method=Speculative Decoding, Speedup=1.65×2026.06 | 54 | 24 | |
| Cascade SDModel=Gemma2 2B→27B, Method=Cascade SD, Speedup=1.81×2026.06 | 54 | 23 | |
| Speculative DecodingModel=LLaMA2 7B→70B, Method=Speculative Decoding, Speedup=1.60×2026.06 | 54 | 27 | |
| Cascade SDModel=LLaMA2 7B→70B, Method=Cascade SD, Speedup=1.80×2026.06 | 54 | 26 | |
| Speculative DecodingModel=Qwen 7B→14B, Method=Speculative Decoding, Speedup=1.58×2026.06 | 54 | 24 | |
| Cascade SDModel=Qwen 7B→14B, Method=Cascade SD, Speedup=1.70×2026.06 | 54 | 23 | |
| VIA-SDModel=Gemma2 2B→9B, Method=VIA-SD (ours), Speedup=2.33×2026.06 | 53 | 13 | |
| Faster CascadesModel=Gemma2 2B→9B, Method=Faster Cascades, Speedup=1.90×2026.06 | 52 | 17 | |
| VIA-SDModel=LLaMA2 7B→13B, Method=VIA-SD (ours), Speedup=2.20×2026.06 | 52 | 16 | |
| Faster CascadesModel=LLaMA2 7B→13B, Method=Faster Cascades, Speedup=1.82×2026.06 | 51 | 20 | |
| Speculative DecodingModel=Gemma2 2B→9B, Method=Speculative Decoding, Speedup=1.55×2026.06 | 50 | 21 | |
| Cascade SDModel=Gemma2 2B→9B, Method=Cascade SD, Speedup=1.62×2026.06 | 50 | 20 | |
| Speculative DecodingModel=LLaMA2 7B→13B, Method=Speculative Decoding, Speedup=1.48×2026.06 | 49 | 26 | |
| Cascade SDModel=LLaMA2 7B→13B, Method=Cascade SD, Speedup=1.60×2026.06 | 49 | 24 |