Speculative Decoding on Spec-Bench OLMo 2 7B
5.12Conversation ScoreEAGLE-3 + SpecVocab
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| EAGLE-3 + SpecVocabBase Model=OLMo 2 7B2026.02 | 5.12 | 3.86 | 4.75 | 4.34 | 5.6 | 4.75 | 4.74 | |
| EAGLE-3 + VocabTrimBase Model=OLMo 2 7B2026.02 | 5.03 | 3.74 | 4.44 | 4.24 | 5.21 | 4.51 | 4.53 | |
| EAGLE-3 (Our Reproduction)Base Model=OLMo 2 7B2026.02 | 4.76 | 3.58 | 4.39 | 4.19 | 5.09 | 4.44 | 4.41 | |
| EAGLE-3 + FR-SpecBase Model=OLMo 2 7B2026.02 | 4.72 | 3.87 | 4.54 | 4.19 | 4.93 | 4.52 | 4.46 | |
| Autoregressive DecodingBase Model=OLMo 2 7B2026.02 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |