Computational Efficiency on GSM8k (Mathematical Reasoning)
6Avg Forward PassesKAVA / PCCoT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| KAVA / PCCoTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 6 | -92 | |
| NO-COT / ICOTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 6.1 | — | |
| NO-COT / ICOTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 6.1 | — | |
| NO-COT / ICOTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 6.2 | — | |
| NO-COT / ICOTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 6.2 | — | |
| KAVA / PCCoTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 6.4 | -80 | |
| KAVA / PCCoTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 6.9 | -78 | |
| KAVA / PCCoTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 7 | -90 | |
| NO-COT / ICOTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 7.4 | — | |
| NO-COT / ICOTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 7.4 | — | |
| KAVA / PCCoTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 9.2 | -89 | |
| KAVA / PCCoTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 9.5 | -76 | |
| CODI / COCONUTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 11.1 | — | |
| CODI / COCONUTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 11.5 | — | |
| CODI / COCONUTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 11.8 | — | |
| CODI / COCONUTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 11.9 | — | |
| CODI / COCONUTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 14 | — | |
| CODI / COCONUTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 14.4 | — | |
| FULL COTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 31.6 | — | |
| FULL COTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 31.9 | — | |
| FULL COTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG2025.10 | 40.4 | — | |
| FULL COTBackbone=LLAMA3.2 - 1B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 71.9 | — | |
| FULL COTBackbone=LLAMA3.2 - 3B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 75.2 | — | |
| FULL COTBackbone=QWEN2.5 - 0.5B - INSTRUCT, Training distribution=GSM8k-AUG-NL2025.10 | 82.4 | — |