Speculative Decoding on SpecBench v1 (test)
4.9WRIT τConFu
Evaluation results
| Method | Links | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ConFuNodes=60, Temperature=0.0, Target Model=Llama3.2-3B-Instruct2026.03 | 4.9 | 2.29 | 4.42 | 2.09 | 5.96 | 2.88 | 5.13 | 2.37 | 5.16 | 2.49 | 4.83 | 2.31 | 4.18 | 1.99 | 4.4 | 2.1 | 4.17 | 1.96 | 4.75 | 2.1 | 5.48 | 2.59 | 4.76 | 2.24 | |
| ConFuNodes=60, Temperature=0.7, Target Model=Llama3.2-3B-Instruct2026.03 | 4.64 | 2.11 | 4.25 | 1.9 | 5.45 | 2.48 | 5.01 | 2.18 | 4.78 | 2.18 | 4.49 | 2.03 | 4.03 | 1.8 | 4.18 | 1.89 | 4.05 | 1.8 | 4.57 | 1.96 | 5.2 | 2.32 | 4.54 | 2.02 | |
| ConFuNodes=30, Temperature=0.0, Target Model=Llama3.2-3B-Instruct2026.03 | 4.55 | 2.18 | 4.18 | 1.96 | 5.72 | 2.75 | 4.96 | 2.27 | 4.85 | 2.33 | 4.59 | 2.18 | 3.95 | 1.88 | 4.16 | 1.98 | 3.91 | 1.83 | 4.52 | 2 | 5.21 | 2.45 | 4.41 | 2.11 | |
| Eagle3Nodes=60, Temperature=0.0, Target Model=Llama3.2-3B-Instruct2026.03 | 4.3 | 2.11 | 3.86 | 1.88 | 5.41 | 2.68 | 4.61 | 2.18 | 4.53 | 2.24 | 4.34 | 2.13 | 3.69 | 1.8 | 3.88 | 1.89 | 3.81 | 1.83 | 4.29 | 1.95 | 4.88 | 2.37 | 4.25 | 2.05 | |
| ConFuNodes=30, Temperature=0.7, Target Model=Llama3.2-3B-Instruct2026.03 | 4.23 | 1.97 | 3.82 | 1.78 | 5.39 | 2.54 | 4.63 | 2.09 | 4.58 | 2.15 | 4.09 | 1.93 | 3.88 | 1.8 | 3.83 | 1.8 | 3.68 | 1.7 | 4.21 | 1.86 | 4.86 | 2.24 | 4.29 | 1.95 | |
| Eagle3Nodes=60, Temperature=0.7, Target Model=Llama3.2-3B-Instruct2026.03 | 4.05 | 1.89 | 3.77 | 1.75 | 5.01 | 2.35 | 4.58 | 2.06 | 4.2 | 1.97 | 3.91 | 1.83 | 3.56 | 1.65 | 3.65 | 1.69 | 3.69 | 1.68 | 4.16 | 1.81 | 4.63 | 2.13 | 4.06 | 1.86 | |
| Eagle3Nodes=30, Temperature=0.0, Target Model=Llama3.2-3B-Instruct2026.03 | 4 | 1.97 | 3.54 | 1.73 | 5.13 | 2.54 | 4.38 | 2.07 | 4.26 | 2.1 | 4.04 | 1.97 | 3.5 | 1.71 | 3.64 | 1.79 | 3.58 | 1.72 | 4.07 | 1.37 | 4.59 | 2.06 | 4 | 1.83 | |
| ConFuNodes=60, Temperature=1.0, Target Model=Llama3.2-3B-Instruct2026.03 | 3.86 | 1.73 | 3.56 | 1.59 | 5.09 | 2.29 | 4.49 | 1.94 | 4.18 | 1.88 | 3.42 | 1.52 | 3.81 | 1.69 | 3.74 | 1.67 | 3.63 | 1.59 | 4.08 | 1.73 | 4.76 | 2.1 | 4.08 | 1.8 | |
| Eagle3Nodes=30, Temperature=0.7, Target Model=Llama3.2-3B-Instruct2026.03 | 3.78 | 1.78 | 3.41 | 1.6 | 4.86 | 2.3 | 4.17 | 1.9 | 4 | 1.89 | 3.75 | 1.75 | 3.44 | 1.6 | 3.4 | 1.6 | 3.43 | 1.6 | 3.9 | 1.29 | 4.32 | 2.01 | 3.81 | 1.7 | |
| ConFuNodes=30, Temperature=1.0, Target Model=Llama3.2-3B-Instruct2026.03 | 3.62 | 1.72 | 3.38 | 1.59 | 5.11 | 2.44 | 4.17 | 1.89 | 3.49 | 1.67 | 3.45 | 1.63 | 3.55 | 1.66 | 3.54 | 1.68 | 3.39 | 1.58 | 3.83 | 1.73 | 4.41 | 2.06 | 3.82 | 1.78 | |
| Eagle3Nodes=60, Temperature=1.0, Target Model=Llama3.2-3B-Instruct2026.03 | 3.47 | 1.67 | 3.08 | 1.48 | 4.37 | 2.11 | 4.26 | 1.97 | 3.36 | 1.62 | 3.15 | 1.51 | 3.37 | 1.6 | 3.28 | 1.56 | 3.34 | 1.57 | 3.62 | 1.65 | 4.12 | 1.95 | 3.6 | 1.7 | |
| Eagle3Nodes=30, Temperature=1.0, Target Model=Llama3.2-3B-Instruct2026.03 | 3.23 | 1.51 | 2.9 | 1.37 | 4.47 | 2.12 | 4.08 | 1.85 | 3.24 | 1.54 | 3.19 | 1.52 | 3.22 | 1.51 | 3.03 | 1.43 | 3.09 | 1.43 | 3.44 | 1.52 | 3.97 | 1.85 | 3.58 | 1.59 |