Language Model Decoding on Spec-Bench
267.6Conv. AccEAGLE-3 + SpecVocab (Ours)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| EAGLE-3 + SpecVocab (Ours)Target Model=Qwen3 8B, Framework=SGLang2026.02 | 267.6 | 166.5 | 216.5 | 249.4 | 313.6 | 257.2 | 245.2 | |
| EAGLE-3 + VocabTrimTarget Model=Qwen3 8B, Framework=SGLang2026.02 | 261.7 | 162 | 207.2 | 239.7 | 302.9 | 252.1 | 237.6 | |
| EAGLE-3 (Our Reproduction)Target Model=Qwen3 8B, Framework=SGLang, Source=Reproduction2026.02 | 258.9 | 159.1 | 203.9 | 244.1 | 297.8 | 246.9 | 235.1 | |
| EAGLE-3 + FR-SpecTarget Model=Qwen3 8B, Framework=SGLang2026.02 | 234.2 | 166 | 218.2 | 230.2 | 289.3 | 241.3 | 229.8 | |
| EAGLE-3Target Model=Qwen3 8B, Framework=SGLang2026.02 | 227.7 | 149.7 | 187.1 | 226.3 | 264.1 | 221.5 | 212.7 | |
| EAGLE-3 + SpecVocabBase Model=OLMo 2 7B, Seeds=52026.02 | 227.4 | 151 | 202.4 | 194 | 248.7 | 204.7 | 204.7 | |
| EAGLE-3 + VocabTrimBase Model=OLMo 2 7B, Seeds=52026.02 | 221.1 | 146 | 188.3 | 187.5 | 229.3 | 193.2 | 194.2 | |
| EAGLE-3Base Model=OLMo 2 7B, Seeds=5, Reproduction=true2026.02 | 209.7 | 140.4 | 185.9 | 185.6 | 224.1 | 190.3 | 189.3 | |
| EAGLE-3 + FR-SpecBase Model=OLMo 2 7B, Seeds=52026.02 | 207.9 | 150.1 | 191.9 | 185 | 217.1 | 193.2 | 190.9 | |
| Autoregressive DecodingTarget Model=Qwen3 8B, Framework=SGLang2026.02 | 98.5 | 92.4 | 96.5 | 98.9 | 98.7 | 97.3 | 97.1 | |
| Autoregressive DecodingBase Model=OLMo 2 7B, Seeds=52026.02 | 94.3 | 88.51 | 92.1 | 94.9 | 95.1 | 92.6 | 92.9 |