LLM Inference Speedup Suite (Alpaca, CodeAlpaca, HumanEval, LiveCodeBench, Math500, MBPP, MT-Bench)
2.61Speedup (Alpaca)DART
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| DARTModel=Qwen14B, Temperature=0, Hardware=NVIDIA A100-40G GPU2026.01 | 2.61 | 3.4 | 2.8 | 2.83 | 2.6 | 3.21 | 2.47 | 2.84 | 3.67 | |
| DARTModel=Qwen8B, Temperature=0, Hardware=NVIDIA A100-40G GPU2026.01 | 2.52 | 3.5 | 3.07 | 2.84 | 2.6 | 3.23 | 2.06 | 2.83 | 3.61 | |
| DARTModel=Qwen4B, Temperature=0, Hardware=NVIDIA A100-40G GPU2026.01 | 2.44 | 3.61 | 2.83 | 2.69 | 2.57 | 3.23 | 2.42 | 2.83 | 3.87 | |
| DARTModel=Qwen1.7B, Temperature=0, Hardware=NVIDIA A100-40G GPU2026.01 | 2.31 | 3.12 | 2.88 | 2.57 | 2.94 | 2.82 | 2.5 | 2.74 | 3.6 | |
| EAGLE3Model=Qwen4B, Temperature=0, Hardware=NVIDIA A100-40G GPU2026.01 | 2.2 | 2.29 | 2.29 | 2.16 | 2.21 | 2.17 | 2.23 | 2.22 | 3.54 | |
| EAGLE3Model=Qwen8B, Temperature=0, Hardware=NVIDIA A100-40G GPU2026.01 | 2.16 | 2.49 | 2.46 | 2.24 | 2.49 | 2.4 | 2.1 | 2.33 | 3.72 | |
| EAGLE3Model=Qwen1.7B, Temperature=0, Hardware=NVIDIA A100-40G GPU2026.01 | 1.91 | 2.26 | 2.32 | 2.28 | 2.46 | 2.19 | 2.09 | 2.21 | 3.8 | |
| EAGLE3Model=Qwen14B, Temperature=0, Hardware=NVIDIA A100-40G GPU2026.01 | 1.76 | 2.19 | 2.4 | 2.32 | 2.54 | 2.14 | 2.01 | 2.19 | 3.48 |