Program-aided math reasoning on TabMWP
75.3AccuracyDeepSeek-Coder-Base
Evaluation results
| Method | Links | |
|---|---|---|
| DeepSeek-Coder-BaseSize=33B, Protocol=Program-Aided Math Reasoning (PAL)2024.01 | 75.3 | |
| CodeLlama-BaseSize=34B, Protocol=Program-Aided Math Reasoning (PAL)2024.01 | 69.8 | |
| DeepSeek-Coder-BaseSize=6.7B, Protocol=Program-Aided Math Reasoning (PAL)2024.01 | 67.9 | |
| CodeLlama-BaseSize=13B, Protocol=Program-Aided Math Reasoning (PAL)2024.01 | 60.3 | |
| CodeLlama-BaseSize=7B, Protocol=Program-Aided Math Reasoning (PAL)2024.01 | 52.9 | |
| StarCoder-BaseSize=16B, Protocol=Program-Aided Math Reasoning (PAL)2024.01 | 45 | |
| CodeGeex-2Size=7B, Protocol=Program-Aided Math Reasoning (PAL)2024.01 | 44.6 | |
| DeepSeek-Coder-BaseSize=1.3B, Protocol=Program-Aided Math Reasoning (PAL)2024.01 | 30 |