Multilingual Mathematical Reasoning on MGSM (Efficiency Metrics)
6.08Speedup (de)FLy
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FLyModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=12025.11 | 6.08 | 17.97 | 6.42 | 18.84 | 6.71 | 18.1 | |
| FLyModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=02025.11 | 5.16 | 14.49 | 5.67 | 15.75 | 6.37 | 16.18 | |
| FLyModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=02025.11 | 2.76 | 11.45 | 2.39 | 11.02 | 2.96 | 12.14 | |
| FLyModel=Llama-3.3-70B-Instruct, Training=Training-free, Temperature=12025.11 | 2.72 | 11.97 | 2.21 | 10.99 | 2.61 | 11.01 | |
| FLyModel=Llama-3.3-70B-Instruct, Training=Training-free, Temperature=02025.11 | 2.64 | 11.41 | 2.18 | 10.29 | 2.6 | 10.58 | |
| FLyModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=12025.11 | 2.56 | 11.77 | 2.35 | 11.27 | 2.71 | 11.63 | |
| SpSModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=02025.11 | 2.47 | 11.01 | 2.63 | 11.22 | 3.01 | 12.51 | |
| TokenRecyclingModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=02025.11 | 2.31 | 3.31 | 2.15 | 3.35 | 2.18 | 3.39 | |
| TokenRecyclingModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=12025.11 | 2.3 | 3.33 | 2.17 | 3.36 | 2.17 | 3.38 | |
| SpSModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=12025.11 | 2.1 | 9.19 | 2.2 | 9.38 | 2.55 | 10.45 | |
| SpSModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=02025.11 | 1.79 | 9.23 | 1.62 | 8.72 | 1.78 | 9.84 | |
| SpSModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=12025.11 | 1.79 | 9.14 | 1.59 | 8.64 | 1.58 | 8.74 | |
| TokenRecyclingModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=12025.11 | 1.73 | 3.22 | 1.79 | 3.34 | 1.78 | 3.34 | |
| TokenRecyclingModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=02025.11 | 1.72 | 3.21 | 1.79 | 3.34 | 1.78 | 3.35 | |
| EAGLE-3Model=Llama-3.3-70B-Instruct, Training=Training-based, Temperature=02025.11 | 1.57 | 2.23 | 1.72 | 2.65 | 0.84 | 1.26 | |
| EAGLE-3Model=Llama-3.3-70B-Instruct, Training=Training-based, Temperature=12025.11 | 1.47 | 2.05 | 1.58 | 2.45 | 0.81 | 1.18 | |
| RESTModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=02025.11 | 1.44 | 1.47 | 1.28 | 1.54 | 1.43 | 1.44 | |
| RESTModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=02025.11 | 1.4 | 1.46 | 1.43 | 1.53 | 1.39 | 1.44 | |
| RESTModel=Llama-3.1-70B-Instruct, Training=Training-free, Temperature=12025.11 | 1.37 | 1.4 | 1.22 | 1.47 | 1.36 | 1.38 | |
| RESTModel=Llama-3.1-405B-Instruct, Training=Training-free, Temperature=12025.11 | 1.31 | 1.36 | 1.33 | 1.43 | 1.3 | 1.34 |