Mathematical Reasoning on GSM-Hard, MultiArith, and SVAMP (Average)
17GSM-Hard AccuracyExplicit CoT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Explicit CoTBackbone=LLAMA-3.2-3B-INSTRUCT2026.06 | 17 | 98.3 | 71 | 62.1 | |
| LOTUS-auxBackbone=LLAMA-3.2-3B-INSTRUCT2026.06 | 16.6 | 99.8 | 74.6 | 63.7 | |
| LOTUS-aux + CODIBackbone=LLAMA-3.2-3B-INSTRUCT2026.06 | 16.5 | 99.4 | 73.9 | 63.3 | |
| LOTUS + CODIBackbone=LLAMA-3.2-3B-INSTRUCT2026.06 | 16.3 | 99.6 | 74.4 | 63.4 | |
| LOTUSBackbone=LLAMA-3.2-3B-INSTRUCT2026.06 | 16 | 99.9 | 75.7 | 63.9 | |
| CODI + SIM-CoTBackbone=LLAMA-3.2-3B-INSTRUCT2026.06 | 14.6 | 98.8 | 74.9 | 62.8 | |
| CODIBackbone=LLAMA-3.2-3B-INSTRUCT2026.06 | 14.3 | 98.7 | 73.3 | 62.1 | |
| Explicit CoTBackbone=LLAMA-3.2-1B-INSTRUCT2026.06 | 13.9 | 96.7 | 65.7 | 58.8 | |
| CODI + SIM-CoTBackbone=LLAMA-3.2-1B-INSTRUCT2026.06 | 12.7 | 96.2 | 61.5 | 56.8 | |
| LOTUSBackbone=LLAMA-3.2-1B-INSTRUCT2026.06 | 12.7 | 98.3 | 60.9 | 57.3 | |
| LOTUS + CODIBackbone=LLAMA-3.2-1B-INSTRUCT2026.06 | 12.7 | 98.9 | 61.1 | 57.6 | |
| LOTUS-auxBackbone=LLAMA-3.2-1B-INSTRUCT2026.06 | 12.6 | 97.8 | 58 | 56.1 | |
| CODIBackbone=LLAMA-3.2-1B-INSTRUCT2026.06 | 11.9 | 95 | 60.6 | 55.8 | |
| LOTUS-aux + CODIBackbone=LLAMA-3.2-1B-INSTRUCT2026.06 | 11.4 | 95.7 | 55 | 54 | |
| LOTUS + CODIBackbone=GPT-22026.06 | 9.8 | 90 | 41.6 | 47.1 | |
| LOTUSBackbone=GPT-22026.06 | 9.5 | 92.4 | 41.8 | 47.9 | |
| No-CoTBackbone=LLAMA-3.2-3B-INSTRUCT2026.06 | 9.5 | 88.7 | 52.9 | 50.4 | |
| CODIBackbone=GPT-22026.06 | 9.4 | 93 | 41.7 | 48 | |
| CODI + SIM-CoTBackbone=GPT-22026.06 | 9.4 | 92.8 | 42.6 | 48.3 | |
| Explicit CoTBackbone=GPT-22026.06 | 9 | 85 | 41.6 | 45.2 | |
| LOTUS-aux + CODIBackbone=GPT-22026.06 | 8.6 | 85.7 | 36.1 | 43.5 | |
| LOTUS-auxBackbone=GPT-22026.06 | 8.2 | 82 | 35.5 | 41.9 | |
| No-CoTBackbone=LLAMA-3.2-1B-INSTRUCT2026.06 | 6.3 | 50.3 | 26.7 | 27.8 | |
| No-CoTBackbone=GPT-22026.06 | 4.3 | 41.1 | 16.4 | 20.6 |