Mathematical Reasoning on SVAMP (Efficiency and Forward Passes)
5.7Avg Forward PassesKAVA / PCCoT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| KAVA / PCCoTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 5.7 | -85 | |
| KAVA / PCCoTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 6 | -65 | |
| NO-COT / ICOTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 6.1 | — | |
| NO-COT / ICOTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 6.1 | — | |
| NO-COT / ICOTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 6.2 | — | |
| NO-COT / ICOTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 6.2 | — | |
| KAVA / PCCoTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 6.4 | -86 | |
| KAVA / PCCoTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 6.5 | -63 | |
| NO-COT / ICOTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 7 | — | |
| NO-COT / ICOTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 7 | — | |
| KAVA / PCCoTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 8.9 | -62 | |
| KAVA / PCCoTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 9 | -80 | |
| CODI / COCONUTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 10.7 | — | |
| CODI / COCONUTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 11 | — | |
| CODI / COCONUTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 11.3 | — | |
| CODI / COCONUTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 11.5 | — | |
| CODI / COCONUTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 13.4 | — | |
| CODI / COCONUTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 14.1 | — | |
| FULL COTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 17 | — | |
| FULL COTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 17.8 | — | |
| FULL COTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 23.3 | — | |
| FULL COTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 38.3 | — | |
| FULL COTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 40.6 | — | |
| FULL COTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 44.9 | — |