Mathematical Reasoning on GSM8K (Tau and SR)
6.92Tau ($ au$)GTO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GTOModel=Vicuna-1.3 13B, Temperature=02025.09 | 6.92 | 5.55 | |
| GTOModel=DeepSeek-R1 Distill-LLaMA 8B, Temperature=02025.09 | 6.92 | 4.2 | |
| GTOModel=DeepSeek-R1 Distill-LLaMA 8B, Temperature=12025.09 | 6.5 | 3.71 | |
| EAGLE-3Model=Vicuna-1.3 13B, Temperature=02025.09 | 6.48 | 4.87 | |
| GTOModel=LLaMA-3.1 Instruct 8B, Temperature=02025.09 | 6.47 | 3.59 | |
| GTOModel=LLaMA-3.3 Instruct 70B, Temperature=02025.09 | 6.25 | 4.11 | |
| EAGLE-3Model=DeepSeek-R1 Distill-LLaMA 8B, Temperature=02025.09 | 6.16 | 3.78 | |
| GTOModel=Vicuna-1.3 13B, Temperature=12025.09 | 6.05 | 4.9 | |
| EAGLE-3Model=LLaMA-3.1 Instruct 8B, Temperature=02025.09 | 6.02 | 3.41 | |
| EAGLE-3Model=LLaMA-3.3 Instruct 70B, Temperature=02025.09 | 5.9 | 3.99 | |
| EAGLE-3Model=DeepSeek-R1 Distill-LLaMA 8B, Temperature=12025.09 | 5.82 | 3.49 | |
| GTOModel=LLaMA-3.3 Instruct 70B, Temperature=12025.09 | 5.82 | 4.07 | |
| EAGLE-3Model=Vicuna-1.3 13B, Temperature=12025.09 | 5.79 | 4.16 | |
| GTOModel=LLaMA-3.1 Instruct 8B, Temperature=12025.09 | 5.75 | 3.02 | |
| EAGLE-3Model=LLaMA-3.3 Instruct 70B, Temperature=12025.09 | 5.65 | 3.88 | |
| GRIFFINModel=DeepSeek-R1 Distill-LLaMA 8B, Temperature=02025.09 | 5.58 | 3.42 | |
| GTOModel=Qwen-3 8B, Temperature=02025.09 | 5.51 | 2.88 | |
| CORALmodel=Llama2-13B-Chat, temperature=0, LM head router=false2025.02 | 5.39 | 3.65 | |
| EAGLE-3Model=LLaMA-3.1 Instruct 8B, Temperature=12025.09 | 5.37 | 2.88 | |
| HASSmodel=Llama2-13B-Chat, temperature=0, LM head router=false2025.02 | 5.32 | 3.6 | |
| CORALmodel=Llama2-13B-Chat, temperature=1, LM head router=false2025.02 | 5.25 | 3.51 | |
| HASSmodel=Llama2-13B-Chat, temperature=1, LM head router=false2025.02 | 5.24 | 3.51 | |
| GRIFFINModel=DeepSeek-R1 Distill-LLaMA 8B, Temperature=12025.09 | 5.23 | 3.13 | |
| GTOModel=Qwen-3 8B, Temperature=12025.09 | 5.17 | 2.63 | |
| CORALmodel=Qwen2.5-14B-Instruct, temperature=0, LM head router=false2025.02 | 5.16 | 3 | |
| GRIFFINModel=LLaMA-3.1 Instruct 8B, Temperature=02025.09 | 5.16 | 3.15 | |
| CORALmodel=Llama2-7B-Chat, temperature=0, LM head router=false2025.02 | 5.12 | 3.25 | |
| CORALmodel=Qwen2.5-14B-Instruct, temperature=1, LM head router=false2025.02 | 5.12 | 2.95 | |
| HASSmodel=Llama2-7B-Chat, temperature=0, LM head router=false2025.02 | 5.11 | 3.25 | |
| CORALmodel=Llama2-7B-Chat, temperature=1, LM head router=false2025.02 | 5.05 | 3.13 | |
| HASSmodel=Qwen2.5-14B-Instruct, temperature=0, LM head router=false2025.02 | 5.03 | 2.92 | |
| EAGLE-3Model=Qwen-3 8B, Temperature=02025.09 | 5.02 | 2.59 | |
| HASSmodel=Llama2-7B-Chat, temperature=1, LM head router=false2025.02 | 4.99 | 3.1 | |
| EAGLE-2model=Llama2-13B-Chat, temperature=0, LM head router=false2025.02 | 4.98 | 3.38 | |
| HASSmodel=Qwen2.5-14B-Instruct, temperature=1, LM head router=false2025.02 | 4.91 | 2.83 | |
| EAGLE-2model=Llama2-13B-Chat, temperature=1, LM head router=false2025.02 | 4.84 | 3.25 | |
| EAGLE-3Model=Qwen-3 8B, Temperature=12025.09 | 4.84 | 2.51 | |
| CORALmodel=Qwen2.5-14B-Instruct, temperature=0, LM head router=true2025.02 | 4.8 | 3.14 | |
| CORALmodel=Qwen2.5-14B-Instruct, temperature=1, LM head router=true2025.02 | 4.72 | 3.05 | |
| EAGLE2Model=Vicuna-1.3 13B, Temperature=02025.09 | 4.72 | 3.83 | |
| EAGLE-2model=Llama2-7B-Chat, temperature=0, LM head router=false2025.02 | 4.7 | 2.98 | |
| CORALmodel=Llama3-8B-Instruct, temperature=0, LM head router=false2025.02 | 4.7 | 2.58 | |
| EAGLE-2model=Llama2-7B-Chat, temperature=1, LM head router=false2025.02 | 4.67 | 2.89 | |
| CORALmodel=Qwen2.5-7B-Instruct, temperature=0, LM head router=false2025.02 | 4.67 | 2.57 | |
| GRIFFINModel=LLaMA-3.1 Instruct 8B, Temperature=12025.09 | 4.67 | 2.66 | |
| EAGLE-2model=Qwen2.5-14B-Instruct, temperature=0, LM head router=false2025.02 | 4.62 | 2.69 | |
| EAGLE-2model=Qwen2.5-14B-Instruct, temperature=1, LM head router=false2025.02 | 4.58 | 2.65 | |
| HASSmodel=Llama3-8B-Instruct, temperature=0, LM head router=false2025.02 | 4.56 | 2.51 | |
| HASSmodel=Qwen2.5-7B-Instruct, temperature=0, LM head router=false2025.02 | 4.53 | 2.49 | |
| CORALmodel=Qwen2.5-7B-Instruct, temperature=1, LM head router=false2025.02 | 4.5 | 2.44 | |
| CORALmodel=Llama3-8B-Instruct, temperature=0, LM head router=true2025.02 | 4.42 | 2.76 | |
| CORALmodel=Llama3-8B-Instruct, temperature=1, LM head router=false2025.02 | 4.39 | 2.38 | |
| CORALmodel=Qwen2.5-7B-Instruct, temperature=0, LM head router=true2025.02 | 4.38 | 2.76 | |
| EAGLE2Model=Vicuna-1.3 13B, Temperature=12025.09 | 4.36 | 3.15 | |
| HASSmodel=Qwen2.5-7B-Instruct, temperature=1, LM head router=false2025.02 | 4.35 | 2.35 | |
| CORALmodel=Llama3-70B-Instruct, temperature=0, LM head router=false2025.02 | 4.34 | 2.9 | |
| EAGLE-2model=Qwen2.5-7B-Instruct, temperature=0, LM head router=false2025.02 | 4.23 | 2.33 | |
| EAGLE-2model=Llama3-8B-Instruct, temperature=0, LM head router=false2025.02 | 4.21 | 2.32 | |
| EAGLE-2Model=LLaMA-3.1 Instruct 8B, Temperature=02025.09 | 4.21 | 2.83 | |
| HASSmodel=Llama3-8B-Instruct, temperature=1, LM head router=false2025.02 | 4.18 | 2.28 | |
| CORALmodel=Qwen2.5-7B-Instruct, temperature=1, LM head router=true2025.02 | 4.16 | 2.58 | |
| HASSmodel=Llama3-70B-Instruct, temperature=0, LM head router=false2025.02 | 4.12 | 2.76 | |
| CORALmodel=Llama3-8B-Instruct, temperature=1, LM head router=true2025.02 | 4.12 | 2.53 | |
| EAGLE-2model=Qwen2.5-7B-Instruct, temperature=1, LM head router=false2025.02 | 4.07 | 2.21 | |
| EAGLE-2model=Llama3-70B-Instruct, temperature=0, LM head router=false2025.02 | 4.03 | 2.7 | |
| EAGLEmodel=Llama2-13B-Chat, temperature=0, LM head router=false2025.02 | 4.01 | 3.1 | |
| EAGLE-2model=Llama3-8B-Instruct, temperature=1, LM head router=false2025.02 | 3.94 | 2.13 | |
| CORALmodel=Llama3-70B-Instruct, temperature=1, LM head router=false2025.02 | 3.91 | 2.61 | |
| EAGLEmodel=Llama2-13B-Chat, temperature=1, LM head router=false2025.02 | 3.84 | 2.83 | |
| EAGLEmodel=Llama2-7B-Chat, temperature=0, LM head router=false2025.02 | 3.84 | 2.73 | |
| HASSmodel=Llama3-70B-Instruct, temperature=1, LM head router=false2025.02 | 3.83 | 2.56 | |
| EAGLEmodel=Llama2-7B-Chat, temperature=1, LM head router=false2025.02 | 3.74 | 2.48 | |
| EAGLE-2model=Llama3-70B-Instruct, temperature=1, LM head router=false2025.02 | 3.73 | 2.5 | |
| EAGLE-2Model=LLaMA-3.1 Instruct 8B, Temperature=12025.09 | 3.71 | 2.37 | |
| EAGLEmodel=Qwen2.5-14B-Instruct, temperature=0, LM head router=false2025.02 | 3.62 | 2.52 | |
| EAGLEModel=Vicuna-1.3 13B, Temperature=02025.09 | 3.62 | 2.74 | |
| EAGLEmodel=Llama3-8B-Instruct, temperature=0, LM head router=false2025.02 | 3.47 | 2.18 | |
| EAGLEmodel=Llama3-70B-Instruct, temperature=0, LM head router=false2025.02 | 3.46 | 2.71 | |
| EAGLEmodel=Qwen2.5-14B-Instruct, temperature=1, LM head router=false2025.02 | 3.46 | 2.33 | |
| HydraModel=Vicuna-1.3 13B, Temperature=02025.09 | 3.37 | 2.61 | |
| HydraModel=Vicuna-1.3 13B, Temperature=12025.09 | 3.37 | 2.35 | |
| EAGLEModel=Vicuna-1.3 13B, Temperature=12025.09 | 3.37 | 2.35 | |
| EAGLEmodel=Qwen2.5-7B-Instruct, temperature=0, LM head router=false2025.02 | 3.32 | 2.05 | |
| EAGLEmodel=Llama3-70B-Instruct, temperature=1, LM head router=false2025.02 | 3.26 | 2.52 | |
| EAGLEmodel=Llama3-8B-Instruct, temperature=1, LM head router=false2025.02 | 3.22 | 1.89 | |
| EAGLEModel=LLaMA-3.1 Instruct 8B, Temperature=02025.09 | 3.06 | 2.04 | |
| EAGLEmodel=Qwen2.5-7B-Instruct, temperature=1, LM head router=false2025.02 | 3 | 1.72 | |
| EAGLEModel=LLaMA-3.1 Instruct 8B, Temperature=12025.09 | 2.89 | 1.92 | |
| MedusaModel=Vicuna-1.3 13B, Temperature=02025.09 | 2.61 | 2.18 | |
| SPSModel=Vicuna-1.3 13B, Temperature=02025.09 | 2 | 1.74 | |
| LookaheadModel=LLaMA-3.1 Instruct 8B, Temperature=02025.09 | 1.93 | 1.84 | |
| PLDModel=LLaMA-3.1 Instruct 8B, Temperature=02025.09 | 1.85 | 1.79 | |
| LookaheadModel=LLaMA-3.3 Instruct 70B, Temperature=02025.09 | 1.82 | 1.73 | |
| SPSModel=Vicuna-1.3 13B, Temperature=12025.09 | 1.75 | 1.47 | |
| LookaheadModel=DeepSeek-R1 Distill-LLaMA 8B, Temperature=02025.09 | 1.68 | 1.62 | |
| PLDModel=LLaMA-3.3 Instruct 70B, Temperature=02025.09 | 1.61 | 1.52 | |
| PLDModel=DeepSeek-R1 Distill-LLaMA 8B, Temperature=02025.09 | 1.54 | 1.48 |