List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM8K (MAT, Speedup)

4.44MAT Score (GSM8K)LogitSpec
1.39282.18392.9753.7661Jul 2, 2025
Updated 6mo ago

Evaluation results

MethodLinks
LogitSpecModels=Llama 2 7B2025.074.443.68
LogitSpecModels=Llama 2 13B2025.074.313.24
PLDModels=Llama 2 7B2025.073.322.98
PLDModels=Llama 2 13B2025.073.242.54
SpSModels=Llama 2 13B2025.072.871.85
SpSModels=Llama 2 7B2025.072.831.87
LogitSpecModels=Llama 2 70B2025.072.371.87
LookaheadModels=Llama 2 13B2025.072.081.52
LookaheadModels=Llama 2 7B2025.072.021.67
SpSModels=Llama 2 70B2025.071.981.69
LookaheadModels=Llama 2 70B2025.071.91.57
RESTModels=Llama 2 70B2025.071.631.32
PLDModels=Llama 2 70B2025.071.631.46
RESTModels=Llama 2 13B2025.071.531.26
RESTModels=Llama 2 7B2025.071.511.18