List your company
ResearchFormerly WizWand

Mathematical Reasoning on MultiArith (Out of Distribution)

100Top-1 Accuracy (Maj@1)SIKeD (All)
52.045664.495376.94589.3947Oct 24, 2024
Updated 8mo ago

Evaluation results

MethodLinks
SIKeD (All)Model=Gemma 7B2024.10100
CombinedModel=Gemma 7B2024.1099.44
SIKeD (Adaptive)Model=Gemma 7B2024.1099.44
CoTModel=Qwen 1.5B2024.1099.44
SIKeD (Adaptive)Model=Qwen 1.5B2024.1099.44
PoTModel=Gemma 7B2024.1098.89
CombinedModel=Qwen 1.5B2024.1098.89
SIKeD (All)Model=Qwen 1.5B2024.1098.89
CoTModel=Gemma 7B2024.1098.33
L2MModel=Qwen 1.5B2024.1096.67
L2MModel=Gemma 7B2024.1096.11
PoTModel=Qwen 1.5B2024.1095.56
SIKeD (Adaptive)Model=Qwen 0.5B2024.1093.33
PoTModel=Qwen 0.5B2024.1092.22
SIKeD (All)Model=Qwen 0.5B2024.1092.22
SIKeD (Adaptive)Model=Gemma 2B2024.1091.11
PoTModel=Gemma 2B2024.1090.56
CombinedModel=Qwen 0.5B2024.1090.56
SIKeD (All)Model=Gemma 2B2024.1088.89
CoTModel=Gemma 2B2024.1087.22
CombinedModel=Gemma 2B2024.1084.44
CoTModel=Qwen 0.5B2024.1083.89
L2MModel=Gemma 2B2024.1081.67
L2MModel=Qwen 0.5B2024.1076.67
SIKeD (Adaptive)Model=SmolLM 1.7B2024.1072.22
SIKeD (All)Model=SmolLM 1.7B2024.1072.22
CombinedModel=SmolLM 1.7B2024.1067.22
PoTModel=SmolLM 1.7B2024.1061.11
CoTModel=SmolLM 1.7B2024.1058.89
L2MModel=SmolLM 1.7B2024.1053.89