Arithmetic Reasoning on SVAMP (Accuracy & Latency)
69.3AccuracyCPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CPOBackbone=Mistral-7B2024.06 | 69.3 | 44.9 | |
| ToTBackbone=Mistral-7B2024.06 | 66 | 4,623.7 | |
| CoTBackbone=Mistral-7B2024.06 | 65.3 | 41.4 | |
| TS-SFTBackbone=Mistral-7B2024.06 | 59 | 41.3 | |
| CPOBackbone=LLAMA2-13B2024.06 | 50 | 48.1 | |
| CPOBackbone=LLAMA2-7B2024.06 | 46 | 32.1 | |
| ToTBackbone=LLAMA2-13B2024.06 | 45.7 | 2,115.3 | |
| TS-SFTBackbone=LLAMA2-13B2024.06 | 44.6 | 46.4 | |
| TS-SFTBackbone=LLAMA2-7B2024.06 | 43.1 | 30.2 | |
| ToTBackbone=LLAMA2-7B2024.06 | 42.7 | 1,861.1 | |
| CoTBackbone=LLAMA2-13B2024.06 | 40.3 | 46.2 | |
| CoTBackbone=LLAMA2-7B2024.06 | 37.7 | 33.3 |