Mathematical Reasoning on GSM8K (Acc, Tp, Spd)
92.6Accuracy (GSM8K)Vanilla Decoding
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Vanilla DecodingModel Series=Llama-3 (70B/1B), Shot=8-shot, Protocol=Pass@12026.04 | 92.6 | 13.4 | 1 | |
| SWIFTModel Series=Llama-3 (70B/1B), Shot=8-shot, Protocol=Pass@12026.04 | 92.6 | 12.9 | 0.96 | |
| LookaheadModel Series=Llama-3 (70B/1B), Shot=8-shot, Protocol=Pass@12026.04 | 92.6 | 11.1 | 0.83 | |
| CSDModel Series=Llama-3 (70B/1B), Shot=8-shot, Protocol=Pass@12026.04 | 92.6 | 24.5 | 1.83 | |
| SpecDecodeModel Series=Llama-3 (70B/1B), Shot=8-shot, Protocol=Pass@12026.04 | 92.3 | 23.3 | 1.74 | |
| Lossy SDModel Series=Llama-3 (70B/1B), Shot=8-shot, Protocol=Pass@1, τ=0.62026.04 | 92.3 | 24.2 | 1.8 | |
| CSDModel Series=Qwen-2.5 (72B/7B), Shot=8-shot, Protocol=Pass@12026.04 | 92.2 | 25.2 | 1.81 | |
| Vanilla DecodingModel Series=Qwen-2.5 (72B/7B), Shot=8-shot, Protocol=Pass@12026.04 | 91.7 | 13.9 | 1 | |
| SpecDecodeModel Series=Qwen-2.5 (72B/7B), Shot=8-shot, Protocol=Pass@12026.04 | 91.5 | 22.2 | 1.59 |