Decoding on Multi-task Evaluation Suite Llama-3.2-1B (test)
394.81MT Throughput (token/s)FR-Spec
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| FR-SpecConfiguration=16k, Temperature=0, Token-frequency statistics=SlimPajama2025.02 | 394.81 | 443 | 326.75 | 476.47 | 394.47 | 375.7 | 402.07 | 401.9 | 1.59 | |
| FR-SpecConfiguration=8k, Temperature=0, Token-frequency statistics=SlimPajama2025.02 | 386.97 | 428.94 | 319.83 | 462.98 | 382.75 | 363.5 | 392.13 | 391.01 | 1.55 | |
| FR-SpecConfiguration=32k, Temperature=0, Token-frequency statistics=SlimPajama2025.02 | 378.9 | 428.75 | 317.68 | 467.53 | 378.39 | 363.7 | 395.95 | 390.13 | 1.55 | |
| FR-SpecConfiguration=64k, Temperature=0, Token-frequency statistics=SlimPajama2025.02 | 349.12 | 406.14 | 297.62 | 427.14 | 350.08 | 338.81 | 390.78 | 365.67 | 1.45 | |
| EAGLE-2Temperature=0, Token-frequency statistics=SlimPajama2025.02 | 306.04 | 358.37 | 266.84 | 372.37 | 305.52 | 294.82 | 360.6 | 323.51 | 1.28 | |
| VanillaTemperature=0, Token-frequency statistics=SlimPajama2025.02 | 259.83 | 255.89 | 220.25 | 263.34 | 260.13 | 248.15 | 256.64 | 252.03 | 1 |