Mathematical Reasoning on MultiArith (Out of Distribution)
100Top-1 Accuracy (Maj@1)SIKeD (All)
Evaluation Results
| Method | Links | |
|---|---|---|
| SIKeD (All)Model=Gemma 7B2024.10 | 100 | |
| CombinedModel=Gemma 7B2024.10 | 99.44 | |
| SIKeD (Adaptive)Model=Gemma 7B2024.10 | 99.44 | |
| CoTModel=Qwen 1.5B2024.10 | 99.44 | |
| SIKeD (Adaptive)Model=Qwen 1.5B2024.10 | 99.44 | |
| PoTModel=Gemma 7B2024.10 | 98.89 | |
| CombinedModel=Qwen 1.5B2024.10 | 98.89 | |
| SIKeD (All)Model=Qwen 1.5B2024.10 | 98.89 | |
| CoTModel=Gemma 7B2024.10 | 98.33 | |
| L2MModel=Qwen 1.5B2024.10 | 96.67 | |
| L2MModel=Gemma 7B2024.10 | 96.11 | |
| PoTModel=Qwen 1.5B2024.10 | 95.56 | |
| SIKeD (Adaptive)Model=Qwen 0.5B2024.10 | 93.33 | |
| PoTModel=Qwen 0.5B2024.10 | 92.22 | |
| SIKeD (All)Model=Qwen 0.5B2024.10 | 92.22 | |
| SIKeD (Adaptive)Model=Gemma 2B2024.10 | 91.11 | |
| PoTModel=Gemma 2B2024.10 | 90.56 | |
| CombinedModel=Qwen 0.5B2024.10 | 90.56 | |
| SIKeD (All)Model=Gemma 2B2024.10 | 88.89 | |
| CoTModel=Gemma 2B2024.10 | 87.22 | |
| CombinedModel=Gemma 2B2024.10 | 84.44 | |
| CoTModel=Qwen 0.5B2024.10 | 83.89 | |
| L2MModel=Gemma 2B2024.10 | 81.67 | |
| L2MModel=Qwen 0.5B2024.10 | 76.67 | |
| SIKeD (Adaptive)Model=SmolLM 1.7B2024.10 | 72.22 | |
| SIKeD (All)Model=SmolLM 1.7B2024.10 | 72.22 | |
| CombinedModel=SmolLM 1.7B2024.10 | 67.22 | |
| PoTModel=SmolLM 1.7B2024.10 | 61.11 | |
| CoTModel=SmolLM 1.7B2024.10 | 58.89 | |
| L2MModel=SmolLM 1.7B2024.10 | 53.89 |