Speculative Decoding Throughput on Spec-Bench
519.7Throughput (Conv.)EAGLE-3 + SpecVocab (Ours)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| EAGLE-3 + SpecVocab (Ours)Backbone=OLMo 2 1B2026.02 | 519.7 | 319 | 494 | 430.7 | 528.2 | 461.3 | 458.8 | |
| EAGLE-3 + VocabTrimBackbone=OLMo 2 1B2026.02 | 502 | 302.2 | 463.7 | 427.6 | 519.7 | 444.7 | 443.3 | |
| EAGLE-3 (Our Reproduction)Backbone=OLMo 2 1B2026.02 | 499.5 | 301.5 | 457.7 | 416 | 501.2 | 445.7 | 436.9 | |
| EAGLE-3 + FR-SpecBackbone=OLMo 2 1B2026.02 | 474 | 315.4 | 477.2 | 419.1 | 500.8 | 438.2 | 437.4 | |
| EAGLE-3 + SpecVocabBackbone=Qwen3 4B2026.02 | 371.5 | 207.9 | 314.8 | 362.3 | 421.2 | 377.4 | 342.5 | |
| EAGLE-3 + VocabTrimBackbone=Qwen3 4B2026.02 | 369.4 | 202.2 | 298.5 | 362 | 410 | 351.6 | 332.3 | |
| EAGLE-3 (Our Reproduction)Backbone=Qwen3 4B2026.02 | 357 | 200.2 | 295.9 | 353.9 | 405.8 | 344.7 | 326.2 | |
| Autoregressive DecodingBackbone=OLMo 2 1B2026.02 | 338.3 | 309 | 330.3 | 340.6 | 339.3 | 332.2 | 331.6 | |
| EAGLE-3 + FR-SpecBackbone=Qwen3 4B2026.02 | 326.1 | 207.5 | 314.3 | 334.1 | 393.9 | 335.3 | 318.5 | |
| Autoregressive DecodingBackbone=Qwen3 4B2026.02 | 160.5 | 143.4 | 157.5 | 161.4 | 160.6 | 159.1 | 157.1 |