Code Generation on HumanEval (MAT and Speedup)
8.53MAT ScoreGraft
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GraftModel=Vicuna-13B2026.05 | 8.53 | 5.41 | |
| EAGLE3-2KModel=Vicuna-13B2026.05 | 8.42 | 4.8 | |
| ECHOModel=Vicuna-13B2026.05 | 8.35 | 5.15 | |
| DDDModel=Vicuna-13B2026.05 | 8.21 | 4.97 | |
| GraftModel=LLaMA3.1-8B2026.05 | 7.28 | 4.28 | |
| EAGLE3-2KModel=LLaMA3.1-8B2026.05 | 7.19 | 3.98 | |
| ECHOModel=LLaMA3.1-8B2026.05 | 7.12 | 4.13 | |
| GraftModel=Qwen3-8B2026.05 | 3.97 | 2.58 | |
| EAGLE3Model=Qwen3-8B2026.05 | 3.91 | 2.28 | |
| ECHOModel=Qwen3-8B2026.05 | 3.85 | 2.44 | |
| GraftModel=Qwen3-32B2026.05 | 3.11 | 2.51 | |
| EAGLE3Model=Qwen3-32B2026.05 | 2.96 | 2.14 | |
| ECHOModel=Qwen3-32B2026.05 | 2.82 | 2.36 | |
| MedusaModel=Vicuna-13B2026.05 | 2.78 | 2.21 | |
| TRModel=Vicuna-13B2026.05 | 2.73 | 2.09 | |
| TRModel=LLaMA3.1-8B2026.05 | 2.56 | 2.03 | |
| SpsModel=Vicuna-13B2026.05 | 2.53 | 1.78 | |
| SAMDModel=Vicuna-13B2026.05 | 2.51 | 2.25 | |
| TRModel=Qwen3-8B2026.05 | 2.45 | 1.97 | |
| TRModel=Qwen3-32B2026.05 | 2.43 | 1.91 | |
| GraftModel=Qwen3-235B2026.05 | 2.42 | 2.24 | |
| EAGLE3-2KModel=Qwen3-235B2026.05 | 2.39 | 1.81 | |
| TRModel=Qwen3-235B2026.05 | 2.31 | 1.85 | |
| ECHOModel=Qwen3-235B2026.05 | 2.3 | 2.13 | |
| PLDModel=Vicuna-13B2026.05 | 1.91 | 1.85 | |
| LookaheadModel=Vicuna-13B2026.05 | 1.75 | 1.64 |