Arithmetic Reasoning on SVAMP latest (test)
64.8Accuracy8-shot
Evaluation Results
| Method | Links | |
|---|---|---|
| 8-shotBackbone=GPT-3.5, Prompting=8-shot2023.12 | 64.8 | |
| SYRELMBackbone=Vicuna 13B2023.12 | 56.65 | |
| PALBackbone=Vicuna 13B2023.12 | 53.7 | |
| ARTBackbone=Vicuna 13B2023.12 | 49.83 | |
| LMFTBackbone=Vicuna 13B, Optimized=Fine-tuned2023.12 | 42.5 | |
| SYRELMBackbone=GPT-J 6B2023.12 | 40.1 | |
| 4-shotBackbone=Vicuna 13B, Prompting=4-shot2023.12 | 37.5 | |
| LMFTBackbone=GPT-J 6B, Optimized=Fine-tuned2023.12 | 31.6 | |
| ToolformerBackbone=GPT-J, Optimized=Fine-tuned2023.12 | 29.4 | |
| 1-shotBackbone=Vicuna 13B, Prompting=1-shot2023.12 | 27 | |
| PALBackbone=GPT-J 6B2023.12 | 22.33 | |
| 4-shotBackbone=GPT-J 6B, Prompting=4-shot2023.12 | 9.45 | |
| ARTBackbone=GPT-J 6B2023.12 | 3.1 | |
| 1-shotBackbone=GPT-J 6B, Prompting=1-shot2023.12 | 3 |