Mathematical Reasoning on GSM8K Hard (Efficiency Metrics)
7.3Average Forward PassesNO-COT / ICOT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| NO-COT / ICOTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 7.3 | — | |
| NO-COT / ICOTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 7.3 | — | |
| NO-COT / ICOTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 7.4 | — | |
| NO-COT / ICOTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 7.4 | — | |
| KAVA / PCCoTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 7.9 | -76 | |
| KAVA / PCCoTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 8.2 | -80 | |
| KAVA / PCCoTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 9.1 | -78 | |
| KAVA / PCCoTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 10 | -88 | |
| NO-COT / ICOTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 10.1 | — | |
| NO-COT / ICOTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 10.1 | — | |
| CODI / COCONUTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 13.1 | — | |
| KAVA / PCCoTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 13.3 | -78 | |
| KAVA / PCCoTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 13.5 | -87 | |
| CODI / COCONUTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 13.9 | — | |
| CODI / COCONUTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 13.9 | — | |
| CODI / COCONUTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 14.2 | — | |
| CODI / COCONUTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 19 | — | |
| CODI / COCONUTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 20.7 | — | |
| FULL COTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 32.9 | — | |
| FULL COTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 40.3 | — | |
| FULL COTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 41.3 | — | |
| FULL COTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 59.6 | — | |
| FULL COTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 80.2 | — | |
| FULL COTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 105.2 | — |