Mathematical Reasoning on MATH500 (Acc, Tp, Spd)
83.2AccuracyCSD
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CSDModel Series=Qwen-2.5 (72B/7B), Shot=4-shot, Protocol=Pass@12026.04 | 83.2 | 32.8 | 2.12 | |
| Vanilla DecodingModel Series=Qwen-2.5 (72B/7B), Shot=4-shot, Protocol=Pass@12026.04 | 82.2 | 15.4 | 1 | |
| SpecDecodeModel Series=Qwen-2.5 (72B/7B), Shot=4-shot, Protocol=Pass@12026.04 | 81.4 | 29.6 | 1.92 | |
| Lossy SDModel Series=Llama-3 (70B/1B), Shot=4-shot, Protocol=Pass@1, τ=0.62026.04 | 49.4 | 30.6 | 2 | |
| CSDModel Series=Llama-3 (70B/1B), Shot=4-shot, Protocol=Pass@12026.04 | 48 | 35.5 | 2.33 | |
| LookaheadModel Series=Llama-3 (70B/1B), Shot=4-shot, Protocol=Pass@12026.04 | 46.6 | 12.5 | 0.82 | |
| SWIFTModel Series=Llama-3 (70B/1B), Shot=4-shot, Protocol=Pass@12026.04 | 46.4 | 16.5 | 1.09 | |
| Vanilla DecodingModel Series=Llama-3 (70B/1B), Shot=4-shot, Protocol=Pass@12026.04 | 46 | 15.2 | 1 | |
| SpecDecodeModel Series=Llama-3 (70B/1B), Shot=4-shot, Protocol=Pass@12026.04 | 45.4 | 28.8 | 1.89 |