List your company
ResearchFormerly WizWand

Arithmetic Reasoning on SVAMP latest (test)

64.8Accuracy8-shot
0.52817.21433.950.586Dec 9, 2023
Updated 8mo ago

Evaluation results

MethodLinks
8-shotBackbone=GPT-3.5, Prompting=8-shot2023.1264.8
SYRELMBackbone=Vicuna 13B2023.1256.65
PALBackbone=Vicuna 13B2023.1253.7
ARTBackbone=Vicuna 13B2023.1249.83
LMFTBackbone=Vicuna 13B, Optimized=Fine-tuned2023.1242.5
SYRELMBackbone=GPT-J 6B2023.1240.1
4-shotBackbone=Vicuna 13B, Prompting=4-shot2023.1237.5
LMFTBackbone=GPT-J 6B, Optimized=Fine-tuned2023.1231.6
ToolformerBackbone=GPT-J, Optimized=Fine-tuned2023.1229.4
1-shotBackbone=Vicuna 13B, Prompting=1-shot2023.1227
PALBackbone=GPT-J 6B2023.1222.33
4-shotBackbone=GPT-J 6B, Prompting=4-shot2023.129.45
ARTBackbone=GPT-J 6B2023.123.1
1-shotBackbone=GPT-J 6B, Prompting=1-shot2023.123