LLM Inference Speedup on Math500
2.84SpeedupDART
Evaluation results
| Method | Links | |
|---|---|---|
| DARTModel=L2 7B, Temperature=02026.01 | 2.84 | |
| HydraModel=L2 7B, Temperature=02026.01 | 2.69 | |
| DARTModel=Qwen1.7B, Temperature=02026.01 | 2.52 | |
| DARTModel=Qwen4B, Temperature=02026.01 | 2.5 | |
| DARTModel=Qwen14B, Temperature=02026.01 | 2.5 | |
| DARTModel=Qwen32B, Temperature=02026.01 | 2.46 | |
| DARTModel=Qwen8B, Temperature=02026.01 | 2.34 | |
| EAGLE3Model=Qwen14B, Temperature=02026.01 | 2.32 | |
| EAGLE3Model=Qwen32B, Temperature=02026.01 | 2.31 | |
| EAGLE3Model=Qwen8B, Temperature=02026.01 | 2.3 | |
| MedusaModel=L2 7B, Temperature=02026.01 | 2.29 | |
| DARTModel=Qwen14B, Temperature=12026.01 | 2.29 | |
| EAGLE3Model=Qwen1.7B, Temperature=02026.01 | 2.25 | |
| DARTModel=Qwen32B, Temperature=12026.01 | 2.16 | |
| EAGLE3Model=Qwen32B, Temperature=12026.01 | 2.15 | |
| EAGLE3Model=Qwen14B, Temperature=12026.01 | 2.1 | |
| EAGLE3Model=Qwen4B, Temperature=02026.01 | 2.06 | |
| LookaheadModel=L2 7B, Temperature=02026.01 | 1.82 | |
| PLDModel=L2 7B, Temperature=02026.01 | 1.59 | |
| SPSModel=Qwen32B, Temperature=02026.01 | 1.06 | |
| SPSModel=Qwen14B, Temperature=02026.01 | 0.96 |