List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM8k (maj1@1)

73.84Accuracy (maj1@1)SIKeD (Adaptive)
4.118422.219240.3258.4208Jul 18, 2023Oct 3, 2023Dec 19, 2023Mar 6, 2024May 22, 2024Aug 7, 2024Oct 24, 2024
Updated 8mo ago

Evaluation results

MethodLinks
SIKeD (Adaptive)Model=Gemma 7B2024.1073.84
SIKeD (All)Model=Gemma 7B2024.1071.42
PoTModel=Gemma 7B2024.1071.34
CombinedModel=Gemma 7B2024.1070.74
L2MModel=Gemma 7B2024.1069.29
CoTModel=Gemma 7B2024.1067.4
SIKeD (Adaptive)Model=Qwen 1.5B2024.1064.97
CombinedModel=Qwen 1.5B2024.1064.44
PoTModel=Qwen 1.5B2024.1064.22
SIKeD (All)Model=Qwen 1.5B2024.1064.14
LLAMA 2Size=70B2023.0756.8
CoTModel=Qwen 1.5B2024.1055.57
L2MModel=Qwen 1.5B2024.1054.59
LLAMA 1Size=65B2023.0750.9
SIKeD (Adaptive)Model=Gemma 2B2024.1047.23
SIKeD (All)Model=Gemma 2B2024.1045.26
PoTModel=Gemma 2B2024.1044.05
CombinedModel=Gemma 2B2024.1044.05
SIKeD (Adaptive)Model=Qwen 0.5B2024.1043.14
SIKeD (All)Model=Qwen 0.5B2024.1043.06
CombinedModel=Qwen 0.5B2024.1042.38
LLAMA 2Size=34B2023.0742.2
PoTModel=Qwen 0.5B2024.1041.62
L2MModel=Gemma 2B2024.1036.92
CoTModel=Gemma 2B2024.1036.54
CoTModel=Qwen 0.5B2024.1036.47
LLAMA 1Size=33B2023.0735.6
L2MModel=Qwen 0.5B2024.1033.59
LLAMA 2Size=13B2023.0728.7
SIKeD (Adaptive)Model=SmolLM 1.7B2024.1027.98
SIKeD (All)Model=SmolLM 1.7B2024.1027.75
CombinedModel=SmolLM 1.7B2024.1024.56
PoTModel=SmolLM 1.7B2024.1023.73
FalconSize=40B2023.0719.6
L2MModel=SmolLM 1.7B2024.1018.73
LLAMA 1Size=13B2023.0717.8
CoTModel=SmolLM 1.7B2024.1016.38
MPTSize=30B2023.0715.2
LLAMA 2Size=7B2023.0714.6
LLAMA 1Size=7B2023.0711
MPTSize=7B2023.076.8
FalconSize=7B2023.076.8