LLM Inference on Aggregate Suite (Alpaca, CodeAlpaca, HumanEval, LiveCodeBench, Math500, MBPP, MT-Bench)
2.87Mean SpeedupDART
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DARTModel=Qwen4B, Temperature=02026.01 | 2.87 | 3.87 | |
| DARTModel=L2 7B, Temperature=02026.01 | 2.85 | 4.08 | |
| DARTModel=Qwen14B, Temperature=02026.01 | 2.77 | 3.67 | |
| DARTModel=Qwen8B, Temperature=02026.01 | 2.71 | 3.61 | |
| HydraModel=L2 7B, Temperature=02026.01 | 2.66 | 3.55 | |
| DARTModel=Qwen1.7B, Temperature=02026.01 | 2.61 | 3.6 | |
| DARTModel=Qwen14B, Temperature=12026.01 | 2.47 | 3.61 | |
| DARTModel=Qwen32B, Temperature=02026.01 | 2.42 | 3.76 | |
| MedusaModel=L2 7B, Temperature=02026.01 | 2.24 | 2.68 | |
| EAGLE3Model=Qwen8B, Temperature=02026.01 | 2.2 | 3.72 | |
| DARTModel=Qwen32B, Temperature=12026.01 | 2.19 | 3.55 | |
| EAGLE3Model=Qwen4B, Temperature=02026.01 | 2.12 | 3.54 | |
| EAGLE3Model=Qwen32B, Temperature=02026.01 | 2.11 | 3.85 | |
| EAGLE3Model=Qwen14B, Temperature=02026.01 | 2.02 | 3.48 | |
| EAGLE3Model=Qwen1.7B, Temperature=02026.01 | 2.01 | 3.8 | |
| EAGLE3Model=Qwen32B, Temperature=12026.01 | 1.97 | 3.67 | |
| EAGLE3Model=Qwen14B, Temperature=12026.01 | 1.89 | 3.38 | |
| PLDModel=L2 7B, Temperature=02026.01 | 1.74 | 1.92 | |
| LookaheadModel=L2 7B, Temperature=02026.01 | 1.61 | 1.81 | |
| SPSModel=Qwen32B, Temperature=02026.01 | 1.09 | 3.45 | |
| SPSModel=Qwen14B, Temperature=02026.01 | 0.98 | 4.17 |