Mathematical Reasoning on GSM8K (Speedup)
2.91SpeedupSlimSpec
Evaluation Results
| Method | Links | |
|---|---|---|
| SlimSpecTarget Model=Llama-3.1-8B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.91 | |
| SpecVocabTarget Model=Llama-3.1-8B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.82 | |
| VocabTrim-TTarget Model=Llama-3.1-8B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.66 | |
| Full VocabTarget Model=Llama-3.1-8B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.42 | |
| SlimSpecTarget Model=Qwen3-30B-A3B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.27 | |
| SpecVocabTarget Model=Qwen3-30B-A3B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.24 | |
| VocabTrim-TTarget Model=Qwen3-30B-A3B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.23 | |
| Full VocabTarget Model=Qwen3-30B-A3B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.12 | |
| SlimSpecTarget Model=GPT-OSS-20B, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.11 | |
| SpecVocabTarget Model=GPT-OSS-20B, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.98 | |
| VocabTrim-TTarget Model=GPT-OSS-20B, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.95 | |
| Full VocabTarget Model=GPT-OSS-20B, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.63 | |
| SlimSpecTarget Model=Llama-3.1-8B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.56 | |
| SlimSpecTarget Model=GPT-OSS-20B, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.56 | |
| SlimSpecTarget Model=Qwen3-30B-A3B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.54 | |
| VocabTrim-TTarget Model=Qwen3-30B-A3B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.53 | |
| SpecVocabTarget Model=Llama-3.1-8B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.52 | |
| VocabTrim-TTarget Model=GPT-OSS-20B, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.52 | |
| SpecVocabTarget Model=Qwen3-30B-A3B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.49 | |
| Full VocabTarget Model=Qwen3-30B-A3B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.48 | |
| VocabTrim-TTarget Model=Llama-3.1-8B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.46 | |
| SpecVocabTarget Model=GPT-OSS-20B, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.45 | |
| Full VocabTarget Model=Llama-3.1-8B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.41 | |
| Full VocabTarget Model=GPT-OSS-20B, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.38 |