Speculative Decoding Evaluation on OOD Mean
5.21SpeedupFLy
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FLyModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=12025.11 | 5.21 | 18.89 | |
| FLyModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=02025.11 | 4.8 | 17.13 | |
| SpSModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=02025.11 | 2.83 | 14.58 | |
| FLyModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=02025.11 | 2.74 | 12.41 | |
| FLyModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=12025.11 | 2.62 | 12.44 | |
| FLyModel=Llama-3.3-70B-Instruct, Training=Training-free, Temperature=12025.11 | 2.56 | 11.75 | |
| FLyModel=Llama-3.3-70B-Instruct, Training=Training-free, Temperature=02025.11 | 2.53 | 11.35 | |
| SpSModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=12025.11 | 2.39 | 12.17 | |
| TokenRecyclingModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=12025.11 | 2.1 | 3.52 | |
| TokenRecyclingModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=02025.11 | 2.08 | 3.47 | |
| SpSModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=02025.11 | 2.04 | 10.91 | |
| SpSModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=12025.11 | 2 | 10.71 | |
| RESTModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=02025.11 | 1.79 | 1.91 | |
| TokenRecyclingModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=02025.11 | 1.68 | 3.31 | |
| TokenRecyclingModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=12025.11 | 1.68 | 3.31 | |
| RESTModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=12025.11 | 1.67 | 1.78 | |
| RESTModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=02025.11 | 1.62 | 1.81 | |
| EAGLE-3Model=Llama-3.3-70B-Instruct, Training=Training-based, Temperature=02025.11 | 1.56 | 2.5 | |
| RESTModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=12025.11 | 1.55 | 1.73 | |
| EAGLE-3Model=Llama-3.3-70B-Instruct, Training=Training-based, Temperature=12025.11 | 1.45 | 2.31 |