Code Generation on HumanEval (MAT, Token/s)
7.54MAT ScoreHedgeSpec
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HedgeSpecTarget Model=Llama-3.1-8B2025.10 | 7.54 | 97.21 | |
| EAGLETarget Model=Llama-3.1-8B2025.10 | 6.77 | 91.98 | |
| HedgeSpecTarget Model=Qwen-3-8B2025.10 | 5.63 | 64.51 | |
| UCBSpecTarget Model=Llama-3.1-8B2025.10 | 5.46 | 72.05 | |
| EXP3SpecTarget Model=Llama-3.1-8B2025.10 | 5.24 | 68.96 | |
| HedgeSpecTarget Model=Qwen-3-32B2025.10 | 5.17 | 33.82 | |
| EAGLETarget Model=Qwen-3-8B2025.10 | 4.84 | 58.81 | |
| UCBSpecTarget Model=Qwen-3-8B2025.10 | 4.3 | 50.59 | |
| EXP3SpecTarget Model=Qwen-3-8B2025.10 | 4.15 | 48.62 | |
| UCBSpecTarget Model=Qwen-3-32B2025.10 | 3.97 | 29.66 | |
| EXP3SpecTarget Model=Qwen-3-32B2025.10 | 3.88 | 28.62 | |
| EAGLETarget Model=Qwen-3-32B2025.10 | 2.97 | 20.35 | |
| Llama-3.1-8BTarget Model=Llama-3.1-8B2025.10 | 1 | 18.67 | |
| Qwen-3-8BTarget Model=Qwen-3-8B2025.10 | 1 | 14.71 | |
| Qwen-3-32BTarget Model=Qwen-3-32B2025.10 | 1 | 8.32 |