Speculative Decoding on ShareGPT Llama-3.1-8B 1.0 (test)
3.2124MT-Bench ScoreEagle_LoRA_ReLU
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Eagle_LoRA_ReLUDecoding Setup=Tree Decoding, Lookahead depth (d)=5, Activation=ReLU2025.12 | 3.2124 | 3.4479 | 3.8806 | 3.1514 | |
| Eagle_LoRA_SiLUDecoding Setup=Tree Decoding, Lookahead depth (d)=5, Activation=SiLU2025.12 | 3.2067 | 3.4849 | 3.8933 | 3.1654 | |
| Eagle_LoRA_GeLUDecoding Setup=Tree Decoding, Lookahead depth (d)=5, Activation=GeLU2025.12 | 3.202 | 3.4891 | 3.8693 | 3.1559 | |
| Eagle_LoRADecoding Setup=Tree Decoding, Lookahead depth (d)=52025.12 | 3.1803 | 3.4765 | 3.8609 | 3.1677 | |
| EagleDecoding Setup=Tree Decoding, Lookahead depth (d)=52025.12 | 3.1406 | 3.4137 | 3.8272 | 3.0851 | |
| Eagle_LoRA_SiLUDecoding Setup=Chain Decoding, Lookahead depth (d)=4, Activation=SiLU2025.12 | 1.7306 | 1.9308 | 2.2981 | 1.6349 | |
| Eagle_LoRA_ReLUDecoding Setup=Chain Decoding, Lookahead depth (d)=4, Activation=ReLU2025.12 | 1.7297 | 1.923 | 2.2711 | 1.6077 | |
| Eagle_LoRA_GeLUDecoding Setup=Chain Decoding, Lookahead depth (d)=4, Activation=GeLU2025.12 | 1.7161 | 1.933 | 2.2841 | 1.5925 | |
| EagleDecoding Setup=Chain Decoding, Lookahead depth (d)=42025.12 | 1.6974 | 1.8861 | 2.2523 | 1.581 | |
| Eagle_LoRADecoding Setup=Chain Decoding, Lookahead depth (d)=42025.12 | 1.6931 | 1.9208 | 2.2758 | 1.6032 |